[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:40 dist-test-slave-csg5 krb5kdc[20530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:40 dist-test-slave-csg5 krb5kdc[20530](info): set up 2 sockets
Aug 07 22:42:40 dist-test-slave-csg5 krb5kdc[20530](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 22:42:41.551541 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:42:41 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:42:41.792611 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46781
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:42.189601 20544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:42.190135 20544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:42.190327 20544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:42.199740 20544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:42.200196 20544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:42.200445 20544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:42.200575 20544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:42.200672 20544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:42.213426 20544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:42.215927 20544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:42.217953 20544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:42.242925 20552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:42.244489 20556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:42.244601 20553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:42.245442 20544 server_base.cc:1034] running on GCE node
I20260807 22:42:42.247310 20544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:42.250209 20544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:42.253680 20544 hybrid_clock.cc:648] HybridClock initialized: now 1786142562253506 us; error 126 us; skew 500 ppm
I20260807 22:42:42.254487 20544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:42 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142562, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:42.260121 20544 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:42:42.263805 20544 webserver.cc:460] Webserver started at http://127.20.10.190:46067/ using document root <none> and password file <none>
I20260807 22:42:42.265410 20544 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:42.265707 20544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:42.266350 20544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:42.272887 20544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "0e0f68f92d6246db9e91e012cacafa1c"
format_stamp: "Formatted at 2026-08-07 22:42:42 on dist-test-slave-csg5"
server_key: "3252e5b9f765494fd6aa5be1e0c6e147"
server_key_iv: "c8ae62a84ea2b3e0e94b9efa6d39f449"
server_key_version: "encryptionkey@0"
I20260807 22:42:42.274258 20544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "0e0f68f92d6246db9e91e012cacafa1c"
format_stamp: "Formatted at 2026-08-07 22:42:42 on dist-test-slave-csg5"
server_key: "3252e5b9f765494fd6aa5be1e0c6e147"
server_key_iv: "c8ae62a84ea2b3e0e94b9efa6d39f449"
server_key_version: "encryptionkey@0"
I20260807 22:42:42.285467 20544 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.001s
I20260807 22:42:42.293449 20563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:42.295994 20544 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260807 22:42:42.296411 20544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "0e0f68f92d6246db9e91e012cacafa1c"
format_stamp: "Formatted at 2026-08-07 22:42:42 on dist-test-slave-csg5"
server_key: "3252e5b9f765494fd6aa5be1e0c6e147"
server_key_iv: "c8ae62a84ea2b3e0e94b9efa6d39f449"
server_key_version: "encryptionkey@0"
I20260807 22:42:42.296763 20544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:42.325814 20544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:42.333280 20544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:42.333750 20544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:42.385488 20544 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46781
I20260807 22:42:42.385699 20618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46781 every 8 connection(s)
I20260807 22:42:42.387550 20544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:42.389889 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 20544
I20260807 22:42:42.390650 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:42:42.395540 20522 external_mini_cluster.cc:1428] Setting key 1878cf93dd4f6365fc8071cbcaeccb6d
I20260807 22:42:42.400746 20619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:42.424707 20619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c: Bootstrap starting.
I20260807 22:42:42.436185 20619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:42.439532 20619 log.cc:824] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:42.446187 20619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c: No bootstrap required, opened a new log
I20260807 22:42:42.461750 20619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } }
I20260807 22:42:42.462736 20619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:42.463002 20619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0f68f92d6246db9e91e012cacafa1c, State: Initialized, Role: FOLLOWER
Aug 07 22:42:42 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:42.464497 20619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } }
I20260807 22:42:42.465193 20619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:42.465720 20619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:42.466143 20619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:42.470636 20619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } }
I20260807 22:42:42.471937 20619 leader_election.cc:302] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e0f68f92d6246db9e91e012cacafa1c; no voters:
I20260807 22:42:42.473693 20619 leader_election.cc:288] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:42.474077 20624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:42.478968 20619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:42.479804 20624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [term 1 LEADER]: Becoming Leader. State: Replica: 0e0f68f92d6246db9e91e012cacafa1c, State: Running, Role: LEADER
I20260807 22:42:42.481323 20624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } }
I20260807 22:42:42.491307 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:42.405582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.20.10.190:46781)
0807 22:42:42.408094 (+ 2512us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:42.408114 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:42.412075 (+ 3961us) server_negotiation.cc:407] Connection header received
0807 22:42:42.432460 (+ 20385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:42.432510 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:42.433727 (+ 1217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:42.434521 (+ 794us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:42.440094 (+ 5573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:42.443170 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:42.446722 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:42.448150 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:42.448553 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:42.467430 (+ 18877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:42.467486 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:42.467523 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:42.467578 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:42.472719 (+ 5141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:42.475004 (+ 2285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:42.475020 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:42.475035 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:42.475540 (+ 505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:42.476588 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:42.476593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:42.476596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:42.477839 (+ 1243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:42.478519 (+ 680us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:42.480457 (+ 1938us) server_negotiation.cc:299] Negotiation successful
0807 22:42:42.481171 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2071,"thread_start_us":1231,"threads_started":1}
I20260807 22:42:42.489494 20626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e0f68f92d6246db9e91e012cacafa1c. Latest consensus state: current_term: 1 leader_uuid: "0e0f68f92d6246db9e91e012cacafa1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } } }
I20260807 22:42:42.490157 20625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e0f68f92d6246db9e91e012cacafa1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0f68f92d6246db9e91e012cacafa1c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46781 } } }
I20260807 22:42:42.493103 20625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:42.493171 20626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:42.501518 20631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:42.515826 20631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:42.539062 20631 catalog_manager.cc:1347] Generated new cluster ID: 0647d061cafb4326b1e8e8138753b376
I20260807 22:42:42.539335 20631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:42.574587 20631 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:42.576421 20631 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:42.590778 20631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e0f68f92d6246db9e91e012cacafa1c: Generated new TSK 0
I20260807 22:42:42.592504 20631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:42:42.670085 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:43.463532 20647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:43.463966 20647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:43.464171 20647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:43.475781 20647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:43.476055 20647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:43.476411 20647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:42:43.489646 20647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:43.491286 20647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:43.493060 20647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:43.515019 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.514870 20654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.516705 20656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.518175 20647 server_base.cc:1034] running on GCE node
I20260807 22:42:43.519223 20647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:43.520601 20647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:43.521951 20647 hybrid_clock.cc:648] HybridClock initialized: now 1786142563521801 us; error 74 us; skew 500 ppm
I20260807 22:42:43.522535 20647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:43 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:43.534991 20647 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:42:43.537600 20647 webserver.cc:460] Webserver started at http://127.20.10.129:41053/ using document root <none> and password file <none>
I20260807 22:42:43.538638 20647 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:43.538918 20647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:43.539587 20647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:43.544009 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "d94bf5132bfa4cc2ab31a3ce0ff3fcff"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-csg5"
server_key: "ae0090938fb9e1d8fdae165a64443ab7"
server_key_iv: "d5d6ef08752f47eb312d85d16f9504f4"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.545260 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "d94bf5132bfa4cc2ab31a3ce0ff3fcff"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-csg5"
server_key: "ae0090938fb9e1d8fdae165a64443ab7"
server_key_iv: "d5d6ef08752f47eb312d85d16f9504f4"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.553575 20647 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260807 22:42:43.559778 20664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.561520 20647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:42:43.561762 20647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "d94bf5132bfa4cc2ab31a3ce0ff3fcff"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-csg5"
server_key: "ae0090938fb9e1d8fdae165a64443ab7"
server_key_iv: "d5d6ef08752f47eb312d85d16f9504f4"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.562081 20647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:43.580917 20647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:43.587705 20647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:43.588145 20647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:43.590469 20647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:43.593181 20647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:43.593374 20647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.593677 20647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:43.593868 20647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.688726 20647 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:46873
I20260807 22:42:43.688784 20787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:46873 every 8 connection(s)
I20260807 22:42:43.690836 20647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:43.693414 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 20647
I20260807 22:42:43.693821 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:43.694844 20522 external_mini_cluster.cc:1428] Setting key 842abab9a593cbf2d7843c704e6e109d
Aug 07 22:42:43 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:42:43.728763 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.696814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:36185 (local address 127.20.10.190:46781)
0807 22:42:43.697895 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.697900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.700204 (+ 2304us) server_negotiation.cc:407] Connection header received
0807 22:42:43.703559 (+ 3355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.703566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.703682 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.703853 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.706902 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.708034 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.710328 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.710978 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.711141 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.717830 (+ 6689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.717841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.717847 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.717922 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.723401 (+ 5479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.724725 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.724742 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.724744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.724936 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.725786 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.725792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.725796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.726384 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.726576 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.727682 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.728019 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":744,"threads_started":1}
I20260807 22:42:43.730000 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.697847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46781 (local address 127.20.10.129:36185)
0807 22:42:43.699977 (+ 2130us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.700039 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.703180 (+ 3141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.704354 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.704375 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.705217 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.706496 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.706524 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.708282 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.708286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.710120 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.710137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.711431 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.711442 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.711803 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.712920 (+ 1117us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.712987 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.717172 (+ 4185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.723635 (+ 6463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.723645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.723663 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.724436 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.725107 (+ 671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.725113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.725117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.725495 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.726567 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.726575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.727137 (+ 562us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.727505 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.728111 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"thread_start_us":951,"threads_started":1}
I20260807 22:42:43.732707 20788 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46781
I20260807 22:42:43.733374 20788 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:43.734711 20788 heartbeater.cc:511] Master 127.20.10.190:46781 requested a full tablet report, sending...
I20260807 22:42:43.739585 20582 ts_manager.cc:194] Registered new tserver with Master: d94bf5132bfa4cc2ab31a3ce0ff3fcff (127.20.10.129:46873)
I20260807 22:42:43.743841 20582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:36185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:42:43.763924 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:44.425704 20796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:44.426101 20796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:44.426338 20796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:44.437695 20796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:44.438099 20796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:44.438385 20796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:42:44.453626 20796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:44.455291 20796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:44.456940 20796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:44.478565 20802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:44.478508 20803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:44.478615 20806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:44.481072 20796 server_base.cc:1034] running on GCE node
I20260807 22:42:44.482134 20796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:44.483625 20796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:44.484994 20796 hybrid_clock.cc:648] HybridClock initialized: now 1786142564484852 us; error 71 us; skew 500 ppm
I20260807 22:42:44.485675 20796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:44 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142564, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:44.491132 20796 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:42:44.493913 20796 webserver.cc:460] Webserver started at http://127.20.10.130:34689/ using document root <none> and password file <none>
I20260807 22:42:44.495157 20796 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:44.495431 20796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:44.495878 20796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:44.499943 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "3ea1fcf6022f44c68adbd2b4d18bee60"
format_stamp: "Formatted at 2026-08-07 22:42:44 on dist-test-slave-csg5"
server_key: "92c9f63f229970c5b3d3c0a7960dee83"
server_key_iv: "e6fc5ea8c8d52ef8b9f366e1e9c414b9"
server_key_version: "encryptionkey@0"
I20260807 22:42:44.501072 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "3ea1fcf6022f44c68adbd2b4d18bee60"
format_stamp: "Formatted at 2026-08-07 22:42:44 on dist-test-slave-csg5"
server_key: "92c9f63f229970c5b3d3c0a7960dee83"
server_key_iv: "e6fc5ea8c8d52ef8b9f366e1e9c414b9"
server_key_version: "encryptionkey@0"
I20260807 22:42:44.510305 20796 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260807 22:42:44.516780 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:44.518882 20796 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260807 22:42:44.519166 20796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "3ea1fcf6022f44c68adbd2b4d18bee60"
format_stamp: "Formatted at 2026-08-07 22:42:44 on dist-test-slave-csg5"
server_key: "92c9f63f229970c5b3d3c0a7960dee83"
server_key_iv: "e6fc5ea8c8d52ef8b9f366e1e9c414b9"
server_key_version: "encryptionkey@0"
I20260807 22:42:44.519548 20796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:44.543272 20796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:44.550289 20796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:44.550952 20796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:44.552781 20796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:44.555613 20796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:44.555815 20796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:44.556027 20796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:44.556134 20796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:44.644304 20796 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35031
I20260807 22:42:44.644708 20932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35031 every 8 connection(s)
I20260807 22:42:44.646100 20796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:44.647192 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 20796
I20260807 22:42:44.647608 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:44.648484 20522 external_mini_cluster.cc:1428] Setting key b8e3dc1508b35aef99f9ea8dbc27c4a9
Aug 07 22:42:44 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142564, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
I20260807 22:42:44.684768 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.650551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44951 (local address 127.20.10.190:46781)
0807 22:42:44.651781 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.651788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.653361 (+ 1573us) server_negotiation.cc:407] Connection header received
0807 22:42:44.658526 (+ 5165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.658535 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.658729 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.658855 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:44.661941 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.663104 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.665270 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.666252 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.666437 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.674460 (+ 8023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:44.674467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:44.674471 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:44.674568 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:44.679548 (+ 4980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.680852 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.680861 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.680868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.681110 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.681881 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.681888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.681894 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.682452 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:44.682646 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.683877 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.684223 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":816,"threads_started":1}
I20260807 22:42:44.686249 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.651605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46781 (local address 127.20.10.130:44951)
0807 22:42:44.653062 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:44.653150 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:44.658051 (+ 4901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:44.659259 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:44.659277 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:44.660608 (+ 1331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:44.661688 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.661713 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.663328 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.663332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.665017 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.665033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.666776 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.666789 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.667234 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.668472 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:44.668524 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:44.673883 (+ 5359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:44.679844 (+ 5961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:44.679854 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:44.679871 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:44.680682 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:44.681385 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:44.681389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:44.681392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:44.681730 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:44.682725 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:44.682739 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:44.683268 (+ 529us) client_negotiation.cc:769] Sending connection context
0807 22:42:44.683661 (+ 393us) client_negotiation.cc:240] Negotiation successful
0807 22:42:44.684158 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":600,"threads_started":1}
I20260807 22:42:44.689077 20933 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46781
I20260807 22:42:44.689577 20933 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:44.691031 20933 heartbeater.cc:511] Master 127.20.10.190:46781 requested a full tablet report, sending...
I20260807 22:42:44.693670 20582 ts_manager.cc:194] Registered new tserver with Master: 3ea1fcf6022f44c68adbd2b4d18bee60 (127.20.10.130:35031)
I20260807 22:42:44.695271 20582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:44951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:42:44.715833 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:42:44.749348 20788 heartbeater.cc:503] Master 127.20.10.190:46781 was elected leader, sending a full tablet report...
W20260807 22:42:45.380633 20940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:45.380993 20940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:45.381299 20940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:45.392292 20940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:45.392517 20940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:45.392742 20940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:42:45.407343 20940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:45.408954 20940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:45.410948 20940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:45.434172 20946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:45.434031 20947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:45.434311 20949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:45.436743 20940 server_base.cc:1034] running on GCE node
I20260807 22:42:45.437709 20940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:45.439224 20940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:45.440533 20940 hybrid_clock.cc:648] HybridClock initialized: now 1786142565440442 us; error 50 us; skew 500 ppm
I20260807 22:42:45.441036 20940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:45.446410 20940 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:42:45.449223 20940 webserver.cc:460] Webserver started at http://127.20.10.131:33353/ using document root <none> and password file <none>
I20260807 22:42:45.450402 20940 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:45.450613 20940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:45.451115 20940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:45.454950 20940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "faea632ed1084018a5fcb2ff2656af7c"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-csg5"
server_key: "b1e9abc39ab11c47e11dc20d2426162c"
server_key_iv: "a9006745a43abb4ce63aedca961ad2ea"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.456015 20940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "faea632ed1084018a5fcb2ff2656af7c"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-csg5"
server_key: "b1e9abc39ab11c47e11dc20d2426162c"
server_key_iv: "a9006745a43abb4ce63aedca961ad2ea"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.463551 20940 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 22:42:45.470041 20956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:45.472414 20940 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260807 22:42:45.472724 20940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "faea632ed1084018a5fcb2ff2656af7c"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-csg5"
server_key: "b1e9abc39ab11c47e11dc20d2426162c"
server_key_iv: "a9006745a43abb4ce63aedca961ad2ea"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.473129 20940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:45.493000 20940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:45.500871 20940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:45.501477 20940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:45.503391 20940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:45.505959 20940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:45.506174 20940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:45.506490 20940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:45.506701 20940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:45.610504 20940 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:44957
I20260807 22:42:45.610771 21074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:44957 every 8 connection(s)
I20260807 22:42:45.613153 20940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:45.619138 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 20940
I20260807 22:42:45.619666 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:45.620793 20522 external_mini_cluster.cc:1428] Setting key 9bc381e9b09b366dcb37e8270e0c3c06
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:45.653244 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.618331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:58911 (local address 127.20.10.190:46781)
0807 22:42:45.619800 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.619807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.623180 (+ 3373us) server_negotiation.cc:407] Connection header received
0807 22:42:45.626789 (+ 3609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.626796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.626993 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.627114 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.630280 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.631948 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.635181 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.636025 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.636163 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.643374 (+ 7211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.643382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.643387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.643469 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.648627 (+ 5158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.649706 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.649712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.649716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.649992 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.650748 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.650753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.650757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.651174 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.651304 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.652361 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.652638 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1234,"thread_start_us":938,"threads_started":1}
I20260807 22:42:45.654749 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.619373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46781 (local address 127.20.10.131:58911)
0807 22:42:45.622965 (+ 3592us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:45.623031 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:45.626402 (+ 3371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:45.627601 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:45.627615 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:45.628633 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:45.629850 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:45.629917 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.632393 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.632398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:45.634925 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:45.634945 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.636558 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.636569 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:45.636943 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.638019 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:45.638059 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:45.642956 (+ 4897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:45.648845 (+ 5889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:45.648863 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:45.648879 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:45.649505 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:45.650166 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:45.650170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:45.650173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:45.650571 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:45.651383 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:45.651390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:45.651821 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 22:42:45.652258 (+ 437us) client_negotiation.cc:240] Negotiation successful
0807 22:42:45.652973 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3081,"thread_start_us":1916,"threads_started":1}
I20260807 22:42:45.656797 21076 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46781
I20260807 22:42:45.657173 21076 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:45.658164 21076 heartbeater.cc:511] Master 127.20.10.190:46781 requested a full tablet report, sending...
I20260807 22:42:45.660297 20582 ts_manager.cc:194] Registered new tserver with Master: faea632ed1084018a5fcb2ff2656af7c (127.20.10.131:44957)
I20260807 22:42:45.661283 20582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:58911
I20260807 22:42:45.668394 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:42:45.700774 20933 heartbeater.cc:503] Master 127.20.10.190:46781 was elected leader, sending a full tablet report...
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:45.739679 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.711289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.20.10.129:46873)
0807 22:42:45.712400 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.712410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.712758 (+ 348us) server_negotiation.cc:407] Connection header received
0807 22:42:45.712931 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.712940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.713207 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.713381 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.716533 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.717779 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.719900 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.720618 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.720788 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.729632 (+ 8844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.729663 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.729682 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.729741 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.735527 (+ 5786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.736744 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.736755 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.736758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.736986 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.737583 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.737587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.737589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.738053 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.738552 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.739001 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.739256 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":500,"threads_started":1}
W20260807 22:42:45.741700 20693 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60088
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:45.804877 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.782749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.20.10.129:46873)
0807 22:42:45.783048 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.783057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.783337 (+ 280us) server_negotiation.cc:407] Connection header received
0807 22:42:45.783596 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.783607 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.783684 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.783806 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.786407 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.787492 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.789492 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.790024 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.790192 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.796984 (+ 6792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.796991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.796994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.797039 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.801161 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.802197 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.802210 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.802213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.802411 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.802940 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.802948 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.802956 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.803282 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.803637 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.803994 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.804299 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:42:45.807242 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 20647
I20260807 22:42:45.829237 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.808697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.20.10.129:46873)
0807 22:42:45.808946 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.808954 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.809579 (+ 625us) server_negotiation.cc:407] Connection header received
0807 22:42:45.809848 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.809852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.809915 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.810024 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.811989 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.812876 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.814641 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.815385 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.815495 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.819266 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.819272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.819276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.819319 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.824322 (+ 5003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.826022 (+ 1700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.826027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.826029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.826396 (+ 367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.827042 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.827046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.827048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.827513 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.827831 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.828340 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.828549 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:42:45.830533 20782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:45.926990 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 20647
I20260807 22:42:45.949503 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 20796
Aug 07 22:42:45 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:42:45.970896 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.951061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.20.10.130:35031)
0807 22:42:45.952106 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.952113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.952142 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:42:45.952225 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.952232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.952447 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.952689 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.954849 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.955939 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.957588 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.958221 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.958334 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.963239 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.963262 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.963277 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.963321 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.966453 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.967386 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.967394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.967397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.967548 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.968328 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.968333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.968335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.968869 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.969398 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.969694 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.970155 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":664,"threads_started":1}
I20260807 22:42:45.972553 20928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:46.069707 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 20796
I20260807 22:42:46.095268 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 20940
Aug 07 22:42:46 dist-test-slave-csg5 krb5kdc[20530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:42:46.115760 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.096576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.20.10.131:44957)
0807 22:42:46.097205 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.097214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.097250 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:42:46.097341 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.097348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.097591 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.097743 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.099448 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.100639 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.102526 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.103339 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.103532 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.108582 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.108602 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.108620 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.108669 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.112289 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.113165 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.113173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.113176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.113326 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.113731 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.113738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.113742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.114148 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.114487 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.115000 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.115337 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":296,"threads_started":1}
I20260807 22:42:46.117202 21069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:46.207273 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 20940
I20260807 22:42:46.230883 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 20544
I20260807 22:42:46.232254 20614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:46.345710 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 20544
2026-08-07T22:42:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6055 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:46 dist-test-slave-csg5 krb5kdc[21099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:46 dist-test-slave-csg5 krb5kdc[21099](info): set up 2 sockets
Aug 07 22:42:46 dist-test-slave-csg5 krb5kdc[21099](info): commencing operation
krb5kdc: starting...
W20260807 22:42:47.591069 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:42:47 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:42:47.819706 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42109
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:42109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:48.201352 21114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:48.201761 21114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:48.201929 21114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:48.209586 21114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:48.209789 21114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:48.209918 21114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:48.210022 21114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:48.210130 21114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:48.220953 21114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:42109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:48.222540 21114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:48.223997 21114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:48.237344 21122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:48.237344 21119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:48.237689 21120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:48.238327 21114 server_base.cc:1034] running on GCE node
I20260807 22:42:48.239275 21114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:48.240926 21114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:48.242187 21114 hybrid_clock.cc:648] HybridClock initialized: now 1786142568242089 us; error 67 us; skew 500 ppm
I20260807 22:42:48.242602 21114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:48 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142568, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:48.246523 21114 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:42:48.248564 21114 webserver.cc:460] Webserver started at http://127.20.10.190:44363/ using document root <none> and password file <none>
I20260807 22:42:48.249377 21114 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:48.249502 21114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:48.249830 21114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:48.253623 21114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6305f774667d430f8a033638f10bebf3"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "52d16804ae76be9e80623f50dd4786ac"
server_key_iv: "e8c4d7a6a5c0fe89d51f8bbc9dd2a5a8"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.254551 21114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6305f774667d430f8a033638f10bebf3"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "52d16804ae76be9e80623f50dd4786ac"
server_key_iv: "e8c4d7a6a5c0fe89d51f8bbc9dd2a5a8"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.260957 21114 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:42:48.265417 21129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:48.267062 21114 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 22:42:48.267413 21114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6305f774667d430f8a033638f10bebf3"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "52d16804ae76be9e80623f50dd4786ac"
server_key_iv: "e8c4d7a6a5c0fe89d51f8bbc9dd2a5a8"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.267676 21114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:48.290581 21114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:48.296052 21114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:48.296530 21114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:48.320128 21114 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:42109
I20260807 22:42:48.320153 21180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:42109 every 8 connection(s)
I20260807 22:42:48.321882 21114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:48.324364 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21114
I20260807 22:42:48.324651 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:42:48.326256 20522 external_mini_cluster.cc:1428] Setting key 78fb422e845c94b4aa48157af76dac86
I20260807 22:42:48.327705 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:42:48 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:48.344969 21181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3: Bootstrap starting.
I20260807 22:42:48.349678 21181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:48.351540 21181 log.cc:824] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:48.356431 21181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3: No bootstrap required, opened a new log
I20260807 22:42:48.357157 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:48.328828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37364 (local address 127.20.10.190:42109)
0807 22:42:48.329718 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:48.329731 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:48.329773 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:42:48.331580 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:48.331606 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:48.332042 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:48.332417 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:48.334876 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:48.336419 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:48.338476 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:48.339437 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:48.339608 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:48.345259 (+ 5651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:48.345271 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:48.345289 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:48.345345 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:48.350438 (+ 5093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:48.351944 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:48.351956 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:48.351966 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:48.352312 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:48.352945 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:48.352951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:48.352954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:48.353871 (+ 917us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:48.354236 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:48.355072 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 22:42:48.355620 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":428,"threads_started":1}
I20260807 22:42:48.367291 21181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } }
I20260807 22:42:48.367775 21181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:48.367899 21181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6305f774667d430f8a033638f10bebf3, State: Initialized, Role: FOLLOWER
I20260807 22:42:48.368590 21181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } }
I20260807 22:42:48.368880 21181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:48.369050 21181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:48.369305 21181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:48.372635 21181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } }
I20260807 22:42:48.373343 21181 leader_election.cc:302] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6305f774667d430f8a033638f10bebf3; no voters:
I20260807 22:42:48.374056 21181 leader_election.cc:288] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:48.374219 21188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:48.376116 21188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [term 1 LEADER]: Becoming Leader. State: Replica: 6305f774667d430f8a033638f10bebf3, State: Running, Role: LEADER
I20260807 22:42:48.376699 21181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:48.376753 21188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } }
I20260807 22:42:48.380096 21189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6305f774667d430f8a033638f10bebf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } } }
I20260807 22:42:48.380509 21189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:48.381101 21190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6305f774667d430f8a033638f10bebf3. Latest consensus state: current_term: 1 leader_uuid: "6305f774667d430f8a033638f10bebf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6305f774667d430f8a033638f10bebf3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42109 } } }
I20260807 22:42:48.381496 21190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:48.381772 21197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:48.388432 21197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:48.400104 21197 catalog_manager.cc:1347] Generated new cluster ID: 570ec01f36b242c79dcaaec5675f4b5a
I20260807 22:42:48.400311 21197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:48.424067 21197 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:48.426328 21197 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:48.434125 21197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6305f774667d430f8a033638f10bebf3: Generated new TSK 0
I20260807 22:42:48.435365 21197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:42:48.503541 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:48.896193 21211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:48.896557 21211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:48.896749 21211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:48.904744 21211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:48.904888 21211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:48.905118 21211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:42:48.915697 21211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:48.917198 21211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:48.918747 21211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:48.936916 21216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:48.937166 21217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:48.937687 21220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:48.938535 21211 server_base.cc:1034] running on GCE node
I20260807 22:42:48.939368 21211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:48.940577 21211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:48.941884 21211 hybrid_clock.cc:648] HybridClock initialized: now 1786142568941778 us; error 39 us; skew 500 ppm
I20260807 22:42:48.942239 21211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:48 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142568, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:48.951102 21211 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:42:48.953040 21211 webserver.cc:460] Webserver started at http://127.20.10.129:34183/ using document root <none> and password file <none>
I20260807 22:42:48.953890 21211 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:48.954020 21211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:48.954432 21211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:48.957964 21211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "0b6e57eb415645b58d640074265cd704"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "ee2b77e7406b0d1507c61221f43d3446"
server_key_iv: "c2076c38abfb583931ec35e62eb3d4a9"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.959062 21211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "0b6e57eb415645b58d640074265cd704"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "ee2b77e7406b0d1507c61221f43d3446"
server_key_iv: "c2076c38abfb583931ec35e62eb3d4a9"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.965593 21211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:42:48.970098 21227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:48.972474 21211 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:42:48.972703 21211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "0b6e57eb415645b58d640074265cd704"
format_stamp: "Formatted at 2026-08-07 22:42:48 on dist-test-slave-csg5"
server_key: "ee2b77e7406b0d1507c61221f43d3446"
server_key_iv: "c2076c38abfb583931ec35e62eb3d4a9"
server_key_version: "encryptionkey@0"
I20260807 22:42:48.972971 21211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:49.014386 21211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:49.020186 21211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:49.020553 21211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:49.021839 21211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:49.023557 21211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:49.023650 21211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:49.023854 21211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:49.023923 21211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:49.068600 21211 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:37227
I20260807 22:42:49.068656 21347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:37227 every 8 connection(s)
I20260807 22:42:49.070079 21211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:49.080386 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21211
I20260807 22:42:49.080650 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:49.081559 20522 external_mini_cluster.cc:1428] Setting key c4015dcd6a41273f2dec380bde171e6c
Aug 07 22:42:49 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142568, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:49.096006 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.073710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54303 (local address 127.20.10.190:42109)
0807 22:42:49.074276 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:49.074281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:49.075249 (+ 968us) server_negotiation.cc:407] Connection header received
0807 22:42:49.077629 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:49.077636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:49.077746 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:49.077893 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:49.079783 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.080659 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.083461 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.084076 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.084217 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.089232 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:49.089240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:49.089248 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:49.089303 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:49.092302 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:49.093275 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:49.093281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:49.093284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:49.093426 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:49.093951 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:49.093955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:49.093958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:49.094265 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:49.094437 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:49.095318 (+ 881us) server_negotiation.cc:299] Negotiation successful
0807 22:42:49.095585 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":310,"threads_started":1}
I20260807 22:42:49.097213 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.074109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42109 (local address 127.20.10.129:54303)
0807 22:42:49.075046 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:49.075104 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:49.077250 (+ 2146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:49.078220 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:49.078229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:49.078940 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:49.079584 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.079603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.080889 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.080897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.083246 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.083265 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.084361 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.084370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.084718 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.085609 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:49.085657 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:49.088902 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:49.092501 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:49.092512 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:49.092534 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:49.093099 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:49.093566 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:49.093572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:49.093575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:49.093807 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:49.094415 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:49.094425 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:49.094902 (+ 477us) client_negotiation.cc:769] Sending connection context
0807 22:42:49.095205 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 22:42:49.095851 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":355,"threads_started":1}
I20260807 22:42:49.099380 21348 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42109
I20260807 22:42:49.099828 21348 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:49.100767 21348 heartbeater.cc:511] Master 127.20.10.190:42109 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:42:49.103962 21146 ts_manager.cc:194] Registered new tserver with Master: 0b6e57eb415645b58d640074265cd704 (127.20.10.129:37227)
I20260807 22:42:49.105794 21146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:54303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:42:49.135939 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:49.652418 21356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:49.652762 21356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:49.652936 21356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:49.660631 21356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:49.660807 21356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:49.660998 21356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:42:49.671845 21356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:49.673410 21356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:49.674932 21356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:49.690361 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:49.690389 21365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:49.690428 21363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:49.691378 21356 server_base.cc:1034] running on GCE node
I20260807 22:42:49.692195 21356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:49.693397 21356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:49.694762 21356 hybrid_clock.cc:648] HybridClock initialized: now 1786142569694674 us; error 65 us; skew 500 ppm
I20260807 22:42:49.695243 21356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:49 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142569, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:49.698993 21356 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:42:49.701054 21356 webserver.cc:460] Webserver started at http://127.20.10.130:36953/ using document root <none> and password file <none>
I20260807 22:42:49.701838 21356 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:49.701996 21356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:49.702361 21356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:49.705744 21356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "8eebd3a5c0314f5dab7adc08d4988ede"
format_stamp: "Formatted at 2026-08-07 22:42:49 on dist-test-slave-csg5"
server_key: "cbaa9fc2636b39d59aec639923e36ceb"
server_key_iv: "01d1db38facd8cad161710ef8e15fa6c"
server_key_version: "encryptionkey@0"
I20260807 22:42:49.706772 21356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "8eebd3a5c0314f5dab7adc08d4988ede"
format_stamp: "Formatted at 2026-08-07 22:42:49 on dist-test-slave-csg5"
server_key: "cbaa9fc2636b39d59aec639923e36ceb"
server_key_iv: "01d1db38facd8cad161710ef8e15fa6c"
server_key_version: "encryptionkey@0"
I20260807 22:42:49.713332 21356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:42:49.717531 21373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:49.719586 21356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:42:49.719851 21356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "8eebd3a5c0314f5dab7adc08d4988ede"
format_stamp: "Formatted at 2026-08-07 22:42:49 on dist-test-slave-csg5"
server_key: "cbaa9fc2636b39d59aec639923e36ceb"
server_key_iv: "01d1db38facd8cad161710ef8e15fa6c"
server_key_version: "encryptionkey@0"
I20260807 22:42:49.720121 21356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:49.739642 21356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:49.745370 21356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:49.745682 21356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:49.746960 21356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:49.748489 21356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:49.748575 21356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:49.748678 21356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:49.748724 21356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:49.788879 21356 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:32853
I20260807 22:42:49.788945 21500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:32853 every 8 connection(s)
I20260807 22:42:49.790371 21356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:49.793845 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21356
I20260807 22:42:49.794114 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:49.794700 20522 external_mini_cluster.cc:1428] Setting key e180b5e8494113ffb0c649b309c946c1
Aug 07 22:42:49 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142569, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:49.814797 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.793761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:39103 (local address 127.20.10.190:42109)
0807 22:42:49.794342 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:49.794351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:49.795577 (+ 1226us) server_negotiation.cc:407] Connection header received
0807 22:42:49.797795 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:49.797803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:49.797922 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:49.798062 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:49.800016 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.801344 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.803865 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.804324 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.804473 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.808338 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:49.808353 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:49.808359 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:49.808399 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:49.811441 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:49.812274 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:49.812280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:49.812282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:49.812410 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:49.812887 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:49.812892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:49.812894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:49.813188 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:49.813356 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:49.814114 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 22:42:49.814365 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":270,"threads_started":1}
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:42:49.815987 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.794349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42109 (local address 127.20.10.130:39103)
0807 22:42:49.795330 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:49.795392 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:49.797396 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:49.798389 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:49.798399 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:49.799134 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:49.799785 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.799806 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.801570 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.801579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.803690 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.803705 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.804599 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.804607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.804851 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.805665 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:49.805689 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:49.808049 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:49.811609 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:49.811619 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:49.811640 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:49.812122 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:49.812531 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:49.812536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:49.812540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:49.812754 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:49.813331 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:49.813340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:49.813715 (+ 375us) client_negotiation.cc:769] Sending connection context
0807 22:42:49.814011 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 22:42:49.814612 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":439,"threads_started":1}
I20260807 22:42:49.818216 21501 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42109
I20260807 22:42:49.818753 21501 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
I20260807 22:42:49.819918 21501 heartbeater.cc:511] Master 127.20.10.190:42109 requested a full tablet report, sending...
I20260807 22:42:49.821939 21146 ts_manager.cc:194] Registered new tserver with Master: 8eebd3a5c0314f5dab7adc08d4988ede (127.20.10.130:32853)
I20260807 22:42:49.822971 21146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:39103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:42:49.847690 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:42:50.109915 21348 heartbeater.cc:503] Master 127.20.10.190:42109 was elected leader, sending a full tablet report...
W20260807 22:42:50.213160 21507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:50.213526 21507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:50.213634 21507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:50.221623 21507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:50.221774 21507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:50.221899 21507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:42:50.232708 21507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:50.234135 21507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:50.235637 21507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:50.253751 21512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:50.253834 21513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:50.253731 21515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:50.255151 21507 server_base.cc:1034] running on GCE node
I20260807 22:42:50.256001 21507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:50.257454 21507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:50.258836 21507 hybrid_clock.cc:648] HybridClock initialized: now 1786142570258796 us; error 69 us; skew 500 ppm
I20260807 22:42:50.259217 21507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:50.263666 21507 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:42:50.265851 21507 webserver.cc:460] Webserver started at http://127.20.10.131:36995/ using document root <none> and password file <none>
I20260807 22:42:50.266806 21507 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:50.266988 21507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:50.267417 21507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:50.271040 21507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "cd1e116628784057836c46ab0e25fee6"
format_stamp: "Formatted at 2026-08-07 22:42:50 on dist-test-slave-csg5"
server_key: "1a2f942f9b1fa51f26785d89dd6acae9"
server_key_iv: "73c03ab0b0d55d9fc44516f2302c99e8"
server_key_version: "encryptionkey@0"
I20260807 22:42:50.272491 21507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "cd1e116628784057836c46ab0e25fee6"
format_stamp: "Formatted at 2026-08-07 22:42:50 on dist-test-slave-csg5"
server_key: "1a2f942f9b1fa51f26785d89dd6acae9"
server_key_iv: "73c03ab0b0d55d9fc44516f2302c99e8"
server_key_version: "encryptionkey@0"
I20260807 22:42:50.279330 21507 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:42:50.284049 21522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:50.285825 21507 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:42:50.286051 21507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "cd1e116628784057836c46ab0e25fee6"
format_stamp: "Formatted at 2026-08-07 22:42:50 on dist-test-slave-csg5"
server_key: "1a2f942f9b1fa51f26785d89dd6acae9"
server_key_iv: "73c03ab0b0d55d9fc44516f2302c99e8"
server_key_version: "encryptionkey@0"
I20260807 22:42:50.286293 21507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:50.306118 21507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:50.311569 21507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:50.311906 21507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:50.312965 21507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:50.314471 21507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:50.314579 21507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:50.314764 21507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:50.314841 21507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:50.362012 21507 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43119
I20260807 22:42:50.362056 21645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43119 every 8 connection(s)
I20260807 22:42:50.363538 21507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:50.371582 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21507
I20260807 22:42:50.371840 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:50.372445 20522 external_mini_cluster.cc:1428] Setting key 3005be05b1358f350c5277a3f740e0c3
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:50.387264 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.366511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:34829 (local address 127.20.10.190:42109)
0807 22:42:50.367144 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.367149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.368050 (+ 901us) server_negotiation.cc:407] Connection header received
0807 22:42:50.370017 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.370023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.370129 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.370272 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.372211 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.373167 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.375223 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.375703 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.375846 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.380057 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.380063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.380067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.380128 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.383460 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.384456 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.384461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.384463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.384606 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.385108 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.385123 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.385125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.385489 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.385620 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.386536 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.386893 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":334,"threads_started":1}
I20260807 22:42:50.389025 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.367084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42109 (local address 127.20.10.131:34829)
0807 22:42:50.367874 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.367919 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.369693 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.370534 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.370543 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.371228 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:50.371975 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.372021 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.373368 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.373373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.375054 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.375069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.375969 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.375978 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.376380 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.377224 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:50.377248 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:50.379700 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:50.383646 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:50.383655 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:50.383670 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:50.384235 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:50.384731 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:50.384735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:50.384737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:50.384967 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:50.385624 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:50.385631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:50.386041 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.386388 (+ 347us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.387439 (+ 1051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":294,"threads_started":1}
I20260807 22:42:50.392093 21646 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42109
I20260807 22:42:50.393131 21646 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:50.394446 21646 heartbeater.cc:511] Master 127.20.10.190:42109 requested a full tablet report, sending...
I20260807 22:42:50.397359 21146 ts_manager.cc:194] Registered new tserver with Master: cd1e116628784057836c46ab0e25fee6 (127.20.10.131:43119)
I20260807 22:42:50.398895 21146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:34829
I20260807 22:42:50.402275 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:42:50.427210 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.410226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.20.10.190:42109)
0807 22:42:50.410514 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.410526 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.411112 (+ 586us) server_negotiation.cc:407] Connection header received
0807 22:42:50.411275 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.411279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.411402 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.411507 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.413204 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.413982 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.415458 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.415976 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.416111 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.419204 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.419210 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.419214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.419268 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.422470 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.423825 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.423831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.423833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.423995 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.424531 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.424535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.424537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.425095 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.426034 (+ 939us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.426495 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.426830 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:42:50.438115 21146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:50.446360 21146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:50.494590 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.466556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.20.10.131:43119)
0807 22:42:50.477758 (+ 11202us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.477774 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.481055 (+ 3281us) server_negotiation.cc:407] Connection header received
0807 22:42:50.481775 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.481785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.482125 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.482373 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:50.483865 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.486443 (+ 2578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.491036 (+ 4593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.493183 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.493351 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.493621 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.493898 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.494183 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10815,"thread_start_us":11135,"threads_started":1}
I20260807 22:42:50.494704 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.466329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43119 (local address 127.0.0.1:43040)
0807 22:42:50.480824 (+ 14495us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.480909 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.481343 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.482919 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.482932 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.482981 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:50.483552 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.483580 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.486769 (+ 3189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.486779 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.490746 (+ 3967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.490772 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.493396 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.493403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.493611 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.493664 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.493804 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.494143 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14127,"thread_start_us":13410,"threads_started":1}
I20260807 22:42:50.496126 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.465533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.20.10.130:32853)
0807 22:42:50.478995 (+ 13462us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.479003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.482024 (+ 3021us) server_negotiation.cc:407] Connection header received
0807 22:42:50.482456 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.482468 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.482836 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.483095 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:50.484991 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.488724 (+ 3733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.492440 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.494086 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.494225 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.494414 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.495626 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.495816 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13242,"thread_start_us":13085,"threads_started":1}
I20260807 22:42:50.496764 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.465173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:32853 (local address 127.0.0.1:49122)
0807 22:42:50.481804 (+ 16631us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.481892 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.482200 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.483637 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.483644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.483681 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:50.484765 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.484781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.488978 (+ 4197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.488986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.492182 (+ 3196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.492203 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.495172 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.495181 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.495398 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.495433 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.496351 (+ 918us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.496515 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15022,"thread_start_us":14901,"threads_started":1}
I20260807 22:42:50.499589 21575 tablet_service.cc:1467] Processing CreateTablet for tablet 2773936ebd594817a7ee8c1db22cfa3b (DEFAULT_TABLE table=test-table [id=7b54ea816bdc41bd8bb8582aaed2eaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:50.500900 21429 tablet_service.cc:1467] Processing CreateTablet for tablet 2773936ebd594817a7ee8c1db22cfa3b (DEFAULT_TABLE table=test-table [id=7b54ea816bdc41bd8bb8582aaed2eaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:50.501579 21575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2773936ebd594817a7ee8c1db22cfa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:50.502338 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2773936ebd594817a7ee8c1db22cfa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:50.502553 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.481756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.20.10.129:37227)
0807 22:42:50.492342 (+ 10586us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.492353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.493376 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 22:42:50.493721 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.493739 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.494073 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.494294 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:50.495669 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.498362 (+ 2693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.500209 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.501290 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.501459 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.501707 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.502039 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.502199 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10335,"thread_start_us":10571,"threads_started":1}
I20260807 22:42:50.502605 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.480996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37227 (local address 127.0.0.1:48978)
0807 22:42:50.493151 (+ 12155us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.493265 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.493508 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.494638 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.494642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.494709 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:50.495454 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.495467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.498544 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.498548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.500028 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.500042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.501515 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.501671 (+ 156us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.501837 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.501863 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.501968 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.502111 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12019,"thread_start_us":11848,"threads_started":1}
I20260807 22:42:50.506196 21278 tablet_service.cc:1467] Processing CreateTablet for tablet 2773936ebd594817a7ee8c1db22cfa3b (DEFAULT_TABLE table=test-table [id=7b54ea816bdc41bd8bb8582aaed2eaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:50.507429 21278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2773936ebd594817a7ee8c1db22cfa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:50.520012 21666 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Bootstrap starting.
I20260807 22:42:50.521200 21665 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Bootstrap starting.
I20260807 22:42:50.522481 21667 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Bootstrap starting.
I20260807 22:42:50.525905 21665 tablet_bootstrap.cc:654] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:50.526180 21666 tablet_bootstrap.cc:654] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:50.526921 21667 tablet_bootstrap.cc:654] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:50.527225 21665 log.cc:824] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:50.527665 21666 log.cc:824] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:50.528294 21667 log.cc:824] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:50.531009 21665 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: No bootstrap required, opened a new log
I20260807 22:42:50.531347 21665 ts_tablet_manager.cc:1397] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 22:42:50.536573 21666 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: No bootstrap required, opened a new log
I20260807 22:42:50.536916 21666 ts_tablet_manager.cc:1397] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260807 22:42:50.536995 21667 tablet_bootstrap.cc:492] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: No bootstrap required, opened a new log
I20260807 22:42:50.537256 21667 ts_tablet_manager.cc:1397] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260807 22:42:50.542079 21665 raft_consensus.cc:348] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.542696 21665 raft_consensus.cc:374] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:50.542830 21665 raft_consensus.cc:729] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eebd3a5c0314f5dab7adc08d4988ede, State: Initialized, Role: FOLLOWER
I20260807 22:42:50.543677 21665 consensus_queue.cc:260] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.547097 21501 heartbeater.cc:503] Master 127.20.10.190:42109 was elected leader, sending a full tablet report...
I20260807 22:42:50.547366 21666 raft_consensus.cc:348] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.548130 21666 raft_consensus.cc:374] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:50.548295 21666 raft_consensus.cc:729] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1e116628784057836c46ab0e25fee6, State: Initialized, Role: FOLLOWER
I20260807 22:42:50.549158 21666 consensus_queue.cc:260] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.550166 21665 ts_tablet_manager.cc:1428] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Time spent starting tablet: real 0.019s user 0.010s sys 0.004s
I20260807 22:42:50.551287 21667 raft_consensus.cc:348] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.552114 21667 raft_consensus.cc:374] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:50.552340 21667 raft_consensus.cc:729] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6e57eb415645b58d640074265cd704, State: Initialized, Role: FOLLOWER
I20260807 22:42:50.553680 21667 consensus_queue.cc:260] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.560066 21646 heartbeater.cc:503] Master 127.20.10.190:42109 was elected leader, sending a full tablet report...
I20260807 22:42:50.561203 21667 ts_tablet_manager.cc:1428] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Time spent starting tablet: real 0.024s user 0.012s sys 0.012s
I20260807 22:42:50.563016 21666 ts_tablet_manager.cc:1428] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Time spent starting tablet: real 0.026s user 0.017s sys 0.009s
W20260807 22:42:50.573283 21349 tablet.cc:2369] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:50.592559 21672 raft_consensus.cc:482] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:50.592924 21672 raft_consensus.cc:504] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.596342 21672 leader_election.cc:288] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eebd3a5c0314f5dab7adc08d4988ede (127.20.10.130:32853), 0b6e57eb415645b58d640074265cd704 (127.20.10.129:37227)
I20260807 22:42:50.604303 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.597102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:45787 (local address 127.20.10.130:32853)
0807 22:42:50.597409 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.597416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.597440 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:42:50.597518 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.597522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.597583 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.597691 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:50.598217 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.599377 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.601667 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.602978 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.603153 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.603236 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.603705 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.603851 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:42:50.604401 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.596259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:32853 (local address 127.20.10.131:45787)
0807 22:42:50.596580 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.596603 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.596749 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.597723 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.597728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.597756 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:50.598054 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.598068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.599551 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.599556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.601439 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.601453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.603154 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.603160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.603533 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.603561 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.603675 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.603820 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":96}
I20260807 22:42:50.606293 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.597068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:38505 (local address 127.20.10.129:37227)
0807 22:42:50.597410 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.597416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.598131 (+ 715us) server_negotiation.cc:407] Connection header received
0807 22:42:50.598378 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.598383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.598451 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.598563 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:50.600183 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.601250 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.603660 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.605152 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.605369 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.605448 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.605821 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.605962 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:42:50.606091 21450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2773936ebd594817a7ee8c1db22cfa3b" candidate_uuid: "cd1e116628784057836c46ab0e25fee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" is_pre_election: true
I20260807 22:42:50.606597 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.596775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37227 (local address 127.20.10.131:38505)
0807 22:42:50.597992 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.598010 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:50.598200 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:50.598578 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:50.598583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:50.598609 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:50.599973 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.599987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.601439 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.601443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.603459 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:50.603471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.605413 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.605420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:50.605631 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.605657 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:42:50.605778 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:42:50.605923 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":21,"thread_start_us":1114,"threads_started":1}
I20260807 22:42:50.607213 21450 raft_consensus.cc:2454] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1e116628784057836c46ab0e25fee6 in term 0.
I20260807 22:42:50.608402 21523 leader_election.cc:302] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eebd3a5c0314f5dab7adc08d4988ede, cd1e116628784057836c46ab0e25fee6; no voters:
I20260807 22:42:50.608556 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2773936ebd594817a7ee8c1db22cfa3b" candidate_uuid: "cd1e116628784057836c46ab0e25fee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6e57eb415645b58d640074265cd704" is_pre_election: true
I20260807 22:42:50.609272 21672 raft_consensus.cc:2781] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:50.609529 21672 raft_consensus.cc:482] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:50.609416 21298 raft_consensus.cc:2454] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1e116628784057836c46ab0e25fee6 in term 0.
I20260807 22:42:50.609751 21672 raft_consensus.cc:3037] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:50.614431 21672 raft_consensus.cc:504] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.615756 21672 leader_election.cc:288] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [CANDIDATE]: Term 1 election: Requested vote from peers 8eebd3a5c0314f5dab7adc08d4988ede (127.20.10.130:32853), 0b6e57eb415645b58d640074265cd704 (127.20.10.129:37227)
W20260807 22:42:50.615746 21647 tablet.cc:2369] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:50.616510 21298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2773936ebd594817a7ee8c1db22cfa3b" candidate_uuid: "cd1e116628784057836c46ab0e25fee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6e57eb415645b58d640074265cd704"
I20260807 22:42:50.616487 21450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2773936ebd594817a7ee8c1db22cfa3b" candidate_uuid: "cd1e116628784057836c46ab0e25fee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eebd3a5c0314f5dab7adc08d4988ede"
I20260807 22:42:50.616854 21298 raft_consensus.cc:3037] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:50.616847 21450 raft_consensus.cc:3037] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:50.620731 21450 raft_consensus.cc:2454] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1e116628784057836c46ab0e25fee6 in term 1.
I20260807 22:42:50.620786 21298 raft_consensus.cc:2454] T 2773936ebd594817a7ee8c1db22cfa3b P 0b6e57eb415645b58d640074265cd704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1e116628784057836c46ab0e25fee6 in term 1.
I20260807 22:42:50.621447 21523 leader_election.cc:302] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6e57eb415645b58d640074265cd704, cd1e116628784057836c46ab0e25fee6; no voters:
I20260807 22:42:50.621959 21672 raft_consensus.cc:2781] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:50.623170 21672 raft_consensus.cc:686] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [term 1 LEADER]: Becoming Leader. State: Replica: cd1e116628784057836c46ab0e25fee6, State: Running, Role: LEADER
I20260807 22:42:50.623991 21672 consensus_queue.cc:237] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } }
I20260807 22:42:50.632467 21146 catalog_manager.cc:5495] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1e116628784057836c46ab0e25fee6 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "cd1e116628784057836c46ab0e25fee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eebd3a5c0314f5dab7adc08d4988ede" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 32853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1e116628784057836c46ab0e25fee6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6e57eb415645b58d640074265cd704" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37227 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:50.689322 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.663613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.20.10.129:37227)
0807 22:42:50.663894 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.663899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.665402 (+ 1503us) server_negotiation.cc:407] Connection header received
0807 22:42:50.665688 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.665694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.665829 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.665953 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.669045 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.670231 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.672336 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.672972 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.673150 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.679677 (+ 6527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.679698 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.679715 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.679762 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.684741 (+ 4979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.685962 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.685972 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.685974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.686349 (+ 375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.686941 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.686946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.686948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.687553 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.688025 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.688451 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.688764 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:50.756975 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.735939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.20.10.129:37227)
0807 22:42:50.736254 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.736259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.736396 (+ 137us) server_negotiation.cc:407] Connection header received
0807 22:42:50.736640 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.736645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.736727 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.736836 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.738432 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.739293 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.741271 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.741789 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.741916 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.748300 (+ 6384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.748316 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.748320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.748377 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.752819 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.753843 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.753848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.753850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.754120 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.755043 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.755048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.755050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.755500 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.755866 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.756181 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.756413 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 22:42:50.793166 21502 tablet.cc:2369] T 2773936ebd594817a7ee8c1db22cfa3b P 8eebd3a5c0314f5dab7adc08d4988ede: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:50.826512 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.800944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.20.10.129:37227)
0807 22:42:50.801193 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.801200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.802384 (+ 1184us) server_negotiation.cc:407] Connection header received
0807 22:42:50.802553 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.802557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.802637 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.802805 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.806552 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.807532 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.810066 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.810717 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.810854 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.817503 (+ 6649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.817510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.817514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.817561 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.821753 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.823016 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.823023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.823026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.823259 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.823748 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.823753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.823757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.824304 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.825040 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.825246 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.825542 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 22:42:50.828985 21255 tablet_service.cc:2908] Rejecting scan request for tablet 2773936ebd594817a7ee8c1db22cfa3b: Uninitialized: clean time has not yet been initialized
I20260807 22:42:50.834389 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21211
I20260807 22:42:50.855997 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.835871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.20.10.129:37227)
0807 22:42:50.836256 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.836264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.836510 (+ 246us) server_negotiation.cc:407] Connection header received
0807 22:42:50.836770 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.836776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.836892 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.837022 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.839577 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.840680 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.842563 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.843384 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.843593 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.847751 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.847757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.847761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.847819 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.852112 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.853176 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.853185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.853193 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.853365 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.853877 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.853883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.853887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.854340 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:50.854877 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:50.855183 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:42:50.855496 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 22:42:50.856922 21343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:50.958783 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21211
I20260807 22:42:50.984045 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21356
Aug 07 22:42:50 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:42:51.001086 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.985043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.20.10.130:32853)
0807 22:42:50.985378 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:50.985386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:50.985414 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:42:50.985473 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:50.985477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:50.985541 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:50.985652 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:50.987032 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.987819 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.989141 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:50.989581 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:50.989729 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:50.994463 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:50.994482 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:50.994498 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:50.994550 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:50.998035 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.998868 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.998876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.998878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.999013 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:50.999436 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:50.999440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:50.999442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:50.999846 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:51.000246 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:51.000478 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 22:42:51.000720 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:42:51.001943 21495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:51.035241 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:51.034613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37227 (local address 127.20.10.131:55741)
0807 22:42:51.034891 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:51.035079 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 22:42:51.035557 21523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37227: connect: Connection refused (error 111)
I20260807 22:42:51.037268 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:51.036875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37227 (local address 127.20.10.131:49877)
0807 22:42:51.037036 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:51.037131 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:42:51.037533 21523 consensus_peers.cc:487] T 2773936ebd594817a7ee8c1db22cfa3b P cd1e116628784057836c46ab0e25fee6 -> Peer 0b6e57eb415645b58d640074265cd704 (127.20.10.129:37227): Couldn't send request to peer 0b6e57eb415645b58d640074265cd704. Status: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:42:51.102583 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21356
I20260807 22:42:51.128782 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21507
Aug 07 22:42:51 dist-test-slave-csg5 krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:42:51.147235 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:51.129706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.20.10.131:43119)
0807 22:42:51.130274 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:51.130279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:51.130305 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:42:51.130364 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:51.130368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:51.130445 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:51.130541 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:51.132161 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:51.132931 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:51.134450 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:51.135021 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:51.135143 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:51.140219 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:51.140237 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:51.140255 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:51.140301 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:51.143747 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:51.144659 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:51.144667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:51.144706 (+ 39us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:51.144908 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:51.145484 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:51.145488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:51.145490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:51.145960 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:51.146219 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:51.146591 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 22:42:51.146918 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":300,"threads_started":1}
I20260807 22:42:51.147993 21641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:51.241384 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21507
I20260807 22:42:51.261590 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21114
I20260807 22:42:51.263043 21176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:51.381768 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21114
2026-08-07T22:42:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5034 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:51 dist-test-slave-csg5 krb5kdc[21697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:51 dist-test-slave-csg5 krb5kdc[21697](info): set up 2 sockets
Aug 07 22:42:51 dist-test-slave-csg5 krb5kdc[21697](info): commencing operation
krb5kdc: starting...
W20260807 22:42:52.580619 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:42:52 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:42:52.832835 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:40883
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:53.197383 21713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:53.197803 21713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:53.198051 21713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:53.207335 21713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:53.207469 21713 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:42:53.207544 21713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:53.207676 21713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:53.207742 21713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:53.207834 21713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:53.219477 21713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:40883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:53.220939 21713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:53.222411 21713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:53.237586 21718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.237545 21721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.237627 21719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:53.238921 21713 server_base.cc:1034] running on GCE node
I20260807 22:42:53.239889 21713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:53.241709 21713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:53.243003 21713 hybrid_clock.cc:648] HybridClock initialized: now 1786142573242922 us; error 52 us; skew 500 ppm
I20260807 22:42:53.243526 21713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:53 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:53.247596 21713 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:42:53.249575 21713 webserver.cc:460] Webserver started at http://127.20.10.190:46447/ using document root <none> and password file <none>
I20260807 22:42:53.250435 21713 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:53.250587 21713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:53.250998 21713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:53.254576 21713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "3abac79c86ec46bebbd433a16a5eaf73"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "366d4b1e29f7a945dcb1d0c2e694fecf"
server_key_iv: "64a5c312e57e8c702c70746652466fbe"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.255671 21713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "3abac79c86ec46bebbd433a16a5eaf73"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "366d4b1e29f7a945dcb1d0c2e694fecf"
server_key_iv: "64a5c312e57e8c702c70746652466fbe"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.262118 21713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:42:53.266494 21729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:53.268244 21713 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 22:42:53.268467 21713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "3abac79c86ec46bebbd433a16a5eaf73"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "366d4b1e29f7a945dcb1d0c2e694fecf"
server_key_iv: "64a5c312e57e8c702c70746652466fbe"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.268735 21713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:53.281998 21713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:53.287009 21713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:53.287343 21713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:53.315311 21713 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:40883
I20260807 22:42:53.315425 21796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:40883 every 8 connection(s)
I20260807 22:42:53.316962 21713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:53.321384 21797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:53.326964 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21713
I20260807 22:42:53.327263 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:42:53.327996 20522 external_mini_cluster.cc:1428] Setting key 1c47613403dd836ff69bfae8ccbed4e5
I20260807 22:42:53.334879 21797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Bootstrap starting.
I20260807 22:42:53.339339 21797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:53.340832 21797 log.cc:824] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:53.344522 21797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: No bootstrap required, opened a new log
Aug 07 22:42:53 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:53.354534 21797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } }
I20260807 22:42:53.355145 21797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:53.355288 21797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abac79c86ec46bebbd433a16a5eaf73, State: Initialized, Role: FOLLOWER
I20260807 22:42:53.356036 21797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } }
I20260807 22:42:53.356343 21797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:53.356501 21797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:53.356726 21797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:53.356768 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:53.330338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.20.10.190:40883)
0807 22:42:53.331405 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:53.331420 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:53.331475 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:42:53.333611 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:53.333641 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:53.334210 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:53.334596 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:53.336803 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.338406 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.340565 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.341622 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.341884 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:53.346808 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:53.346819 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:53.346838 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:53.346887 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:53.351255 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:53.352192 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:53.352201 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:53.352209 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:53.352437 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:53.352929 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:53.352933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:53.352935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:53.353614 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:53.354017 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:53.354560 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 22:42:53.355184 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":515,"threads_started":1}
I20260807 22:42:53.360199 21797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } }
I20260807 22:42:53.360898 21797 leader_election.cc:302] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3abac79c86ec46bebbd433a16a5eaf73; no voters:
I20260807 22:42:53.361555 21797 leader_election.cc:288] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:53.361733 21802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:53.364184 21797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:53.364312 21802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [term 1 LEADER]: Becoming Leader. State: Replica: 3abac79c86ec46bebbd433a16a5eaf73, State: Running, Role: LEADER
I20260807 22:42:53.365063 21802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } }
I20260807 22:42:53.368695 21803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3abac79c86ec46bebbd433a16a5eaf73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } } }
I20260807 22:42:53.369175 21803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:53.369728 21804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3abac79c86ec46bebbd433a16a5eaf73. Latest consensus state: current_term: 1 leader_uuid: "3abac79c86ec46bebbd433a16a5eaf73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3abac79c86ec46bebbd433a16a5eaf73" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 40883 } } }
I20260807 22:42:53.370075 21804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:53.371659 21812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:53.377797 21812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:53.388415 21812 catalog_manager.cc:1347] Generated new cluster ID: dba0bf1e00e14da08946f7a6789a8288
I20260807 22:42:53.388581 21812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:53.419265 21812 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:53.421505 21812 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:53.432477 21812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Generated new TSK 0
I20260807 22:42:53.433770 21812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:42:53.501205 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:40883
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:53.896093 21824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:53.896427 21824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:53.896605 21824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:53.904592 21824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:53.904748 21824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:53.904907 21824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:53.905056 21824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:42:53.915635 21824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:53.917178 21824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:53.918706 21824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:53.933480 21829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.933486 21830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.933943 21833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:53.935168 21824 server_base.cc:1034] running on GCE node
I20260807 22:42:53.936028 21824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:53.937100 21824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:53.938382 21824 hybrid_clock.cc:648] HybridClock initialized: now 1786142573938314 us; error 50 us; skew 500 ppm
I20260807 22:42:53.938798 21824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:53 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:53.942529 21824 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:42:53.944401 21824 webserver.cc:460] Webserver started at http://127.20.10.129:44663/ using document root <none> and password file <none>
I20260807 22:42:53.945209 21824 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:53.945364 21824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:53.945791 21824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:53.949160 21824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "842471cbc49f44199a5ff0d298552a64"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "0b607af798c883858ce180b928e45d32"
server_key_iv: "b5c4a6e8e08344ae319d1876a93dc79b"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.950084 21824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "842471cbc49f44199a5ff0d298552a64"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "0b607af798c883858ce180b928e45d32"
server_key_iv: "b5c4a6e8e08344ae319d1876a93dc79b"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.956200 21824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:42:53.960624 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:53.962285 21824 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260807 22:42:53.962538 21824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "842471cbc49f44199a5ff0d298552a64"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-csg5"
server_key: "0b607af798c883858ce180b928e45d32"
server_key_iv: "b5c4a6e8e08344ae319d1876a93dc79b"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.962877 21824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:53.985881 21824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:53.991020 21824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:53.991369 21824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:53.994692 21824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:53.994814 21824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 22:42:53.995026 21824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:53.995118 21824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:54 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:54.019771 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:53.995131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:45093 (local address 127.20.10.190:40883)
0807 22:42:53.995753 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:53.995760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:53.997127 (+ 1367us) server_negotiation.cc:407] Connection header received
0807 22:42:53.999379 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:53.999384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:53.999484 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:53.999607 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.002008 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.002863 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.005482 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.005923 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.006066 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.011661 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.011667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.011670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:54.011710 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.015032 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.016500 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.016509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.016513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.016773 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.017538 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.017546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.017550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.017890 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.018075 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.018997 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.019301 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":360,"threads_started":1}
I20260807 22:42:54.021063 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:53.995986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.129:45093)
0807 22:42:53.996951 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:53.996993 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:53.999064 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:53.999903 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:53.999919 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:54.000831 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:54.001786 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.001817 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.003061 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.003066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.005305 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.005321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.006184 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.006192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.006450 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.007347 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:54.007390 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:54.011374 (+ 3984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:54.015371 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.015392 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.015421 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.016317 (+ 896us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.016979 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.016986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.016993 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.017399 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.018045 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:54.018055 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:54.018503 (+ 448us) client_negotiation.cc:769] Sending connection context
0807 22:42:54.018891 (+ 388us) client_negotiation.cc:240] Negotiation successful
0807 22:42:54.019593 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":378,"threads_started":1}
I20260807 22:42:54.054579 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.031910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54959 (local address 127.20.10.190:40883)
0807 22:42:54.032160 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:54.032169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:54.033429 (+ 1260us) server_negotiation.cc:407] Connection header received
0807 22:42:54.033569 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:54.033579 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:54.033715 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:54.033841 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.036253 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.037586 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.040997 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.041837 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.042013 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.046054 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.046065 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.046073 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:54.046133 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.050092 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.051654 (+ 1562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.051663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.051669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.051893 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.052544 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.052553 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.052558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.053075 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.053264 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.054083 (+ 819us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.054321 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 22:42:54.055493 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.032010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.129:54959)
0807 22:42:54.033262 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:54.033288 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:54.033448 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:54.034130 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:54.034140 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:54.035217 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:54.035892 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.035924 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.037840 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.037855 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.040633 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.040665 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.042250 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.042263 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.042622 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.043647 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:54.043684 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:54.045618 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:54.050326 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.050341 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.050350 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.051447 (+ 1097us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.052058 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.052067 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.052075 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.052332 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.053231 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:54.053236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:54.053657 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 22:42:54.053932 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 22:42:54.054506 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":924,"threads_started":1}
I20260807 22:42:54.059054 21824 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:46345
I20260807 22:42:54.059116 21973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:46345 every 8 connection(s)
I20260807 22:42:54.060640 21824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:54.063149 21974 heartbeater.cc:347] Connected to a master server at 127.20.10.190:40883
I20260807 22:42:54.063372 21974 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:54.064030 21974 heartbeater.cc:511] Master 127.20.10.190:40883 requested a full tablet report, sending...
I20260807 22:42:54.066147 21747 ts_manager.cc:194] Registered new tserver with Master: 842471cbc49f44199a5ff0d298552a64 (127.20.10.129:46345)
I20260807 22:42:54.067868 21747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:45093
I20260807 22:42:54.069036 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21824
I20260807 22:42:54.069309 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:54.070195 20522 external_mini_cluster.cc:1428] Setting key 214a50ddb2e2a9afa6cbaa9302ce7718
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:42:54.129673 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:40883
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:54.518632 21978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:54.519088 21978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:54.519258 21978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:54.527166 21978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:54.527324 21978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:54.527489 21978 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:54.527628 21978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:42:54.538648 21978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:54.540201 21978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:54.541695 21978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:54.556770 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:54.556685 21984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:54.556706 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:54.557694 21978 server_base.cc:1034] running on GCE node
I20260807 22:42:54.558435 21978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:54.559631 21978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:54.560940 21978 hybrid_clock.cc:648] HybridClock initialized: now 1786142574560855 us; error 48 us; skew 500 ppm
I20260807 22:42:54.561353 21978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:54 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:54.565068 21978 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:42:54.567070 21978 webserver.cc:460] Webserver started at http://127.20.10.130:39097/ using document root <none> and password file <none>
I20260807 22:42:54.567924 21978 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:54.568043 21978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:54.568444 21978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:54.571805 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "f9c1ea792d154736b29693b75acf0a97"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-csg5"
server_key: "ee47e4531ddd53f07a9b457c5ef8629c"
server_key_iv: "9e7daf6092765185ab9ff999adf9396c"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.572813 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "f9c1ea792d154736b29693b75acf0a97"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-csg5"
server_key: "ee47e4531ddd53f07a9b457c5ef8629c"
server_key_iv: "9e7daf6092765185ab9ff999adf9396c"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.579082 21978 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:42:54.583088 21994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:54.584695 21978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:42:54.584944 21978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "f9c1ea792d154736b29693b75acf0a97"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-csg5"
server_key: "ee47e4531ddd53f07a9b457c5ef8629c"
server_key_iv: "9e7daf6092765185ab9ff999adf9396c"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.585207 21978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:54.603077 21978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:54.609447 21978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:54.609853 21978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:54.614141 21978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:54.614275 21978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:54.614526 21978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:54.614645 21978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:54 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:54.646130 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.614997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:37003 (local address 127.20.10.190:40883)
0807 22:42:54.617705 (+ 2708us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:54.617713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:54.617742 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:42:54.620064 (+ 2322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:54.620073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:54.620196 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:54.620339 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.623244 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.625078 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.627613 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.628343 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.628542 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.636169 (+ 7627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.636179 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.636187 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:54.636258 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.641240 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.642832 (+ 1592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.642839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.642842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.643053 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.643944 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.643952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.643958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.644373 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.644563 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.645402 (+ 839us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.645675 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":2395,"threads_started":1}
I20260807 22:42:54.647827 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.615604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.130:37003)
0807 22:42:54.616649 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:54.616706 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:54.619676 (+ 2970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:54.620643 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:54.620654 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:54.621643 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:54.622870 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.622896 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.625391 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.625400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.627306 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.627337 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.628941 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.628947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.629451 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.630587 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:54.630616 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:54.635711 (+ 5095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:54.641559 (+ 5848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.641568 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.641585 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.642551 (+ 966us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.643247 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.643255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.643266 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.643757 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.644526 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:54.644536 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:54.644981 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 22:42:54.645302 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 22:42:54.645920 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":466,"threads_started":1}
I20260807 22:42:54.677037 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.657504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44829 (local address 127.20.10.190:40883)
0807 22:42:54.657788 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:54.657798 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:54.658505 (+ 707us) server_negotiation.cc:407] Connection header received
0807 22:42:54.658825 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:54.658833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:54.658944 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:54.659071 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.660968 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.662336 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.664921 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.665753 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.665924 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.668934 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.668940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.668944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:54.669027 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.672981 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.674413 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.674421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.674424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.674628 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.675212 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.675218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.675222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.675626 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.675828 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.676324 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.676607 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:42:54.677160 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.657485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.130:44829)
0807 22:42:54.658349 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:54.658379 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:54.658586 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:54.659145 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:54.659154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:54.660082 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:54.660695 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.660716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.662584 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.662592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.664671 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:54.664689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.665997 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.666006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:54.666263 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.667141 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:54.667174 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:54.668685 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:54.673240 (+ 4555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.673247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.673252 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.674131 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.674806 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:54.674819 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:54.674823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:54.675052 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:54.675772 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:54.675778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:54.676009 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 22:42:54.676188 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 22:42:54.676609 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":433,"threads_started":1}
I20260807 22:42:54.682734 21978 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35699
I20260807 22:42:54.682826 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35699 every 8 connection(s)
I20260807 22:42:54.684085 21978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:54.685364 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 21978
I20260807 22:42:54.685624 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:54.686156 20522 external_mini_cluster.cc:1428] Setting key c46dce7937f779da50b16f5674d248b6
I20260807 22:42:54.686872 22124 heartbeater.cc:347] Connected to a master server at 127.20.10.190:40883
I20260807 22:42:54.687184 22124 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:54.688277 22124 heartbeater.cc:511] Master 127.20.10.190:40883 requested a full tablet report, sending...
I20260807 22:42:54.690894 21747 ts_manager.cc:194] Registered new tserver with Master: f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699)
I20260807 22:42:54.692262 21747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:37003
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:42:54.747458 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:40883
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:42:55.072036 21974 heartbeater.cc:503] Master 127.20.10.190:40883 was elected leader, sending a full tablet report...
W20260807 22:42:55.115491 22128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:55.115948 22128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:55.116125 22128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:55.124714 22128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:55.124922 22128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:55.125034 22128 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:55.125144 22128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:42:55.136863 22128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:55.138279 22128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:55.139928 22128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:55.156005 22135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:55.155915 22140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:55.155975 22138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:55.156893 22128 server_base.cc:1034] running on GCE node
I20260807 22:42:55.157624 22128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:55.158833 22128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:55.160128 22128 hybrid_clock.cc:648] HybridClock initialized: now 1786142575160029 us; error 46 us; skew 500 ppm
I20260807 22:42:55.160568 22128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:55 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:55.164318 22128 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:42:55.166340 22128 webserver.cc:460] Webserver started at http://127.20.10.131:39057/ using document root <none> and password file <none>
I20260807 22:42:55.167296 22128 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:55.167455 22128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:55.167903 22128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:55.171736 22128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "4e1597e0ad794496836832051f61bcb9"
format_stamp: "Formatted at 2026-08-07 22:42:55 on dist-test-slave-csg5"
server_key: "2c22da3cb5791fdd364d433a2341f27d"
server_key_iv: "6c7ca6a12e9feb27cb411c96f3e82e37"
server_key_version: "encryptionkey@0"
I20260807 22:42:55.173008 22128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "4e1597e0ad794496836832051f61bcb9"
format_stamp: "Formatted at 2026-08-07 22:42:55 on dist-test-slave-csg5"
server_key: "2c22da3cb5791fdd364d433a2341f27d"
server_key_iv: "6c7ca6a12e9feb27cb411c96f3e82e37"
server_key_version: "encryptionkey@0"
I20260807 22:42:55.180197 22128 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 22:42:55.185515 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:55.187564 22128 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:42:55.187796 22128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "4e1597e0ad794496836832051f61bcb9"
format_stamp: "Formatted at 2026-08-07 22:42:55 on dist-test-slave-csg5"
server_key: "2c22da3cb5791fdd364d433a2341f27d"
server_key_iv: "6c7ca6a12e9feb27cb411c96f3e82e37"
server_key_version: "encryptionkey@0"
I20260807 22:42:55.188072 22128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:55.209250 22128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:55.216481 22128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:55.216854 22128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:55.220747 22128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:55.220898 22128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:55.221110 22128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:55.221205 22128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:55 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:55.252259 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.221051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:37247 (local address 127.20.10.190:40883)
0807 22:42:55.221700 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.221711 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.223264 (+ 1553us) server_negotiation.cc:407] Connection header received
0807 22:42:55.227244 (+ 3980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.227251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.227373 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.227517 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:55.230887 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.232099 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.236288 (+ 4189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.237013 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.237185 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.242521 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:55.242528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:55.242531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:55.242574 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:55.247059 (+ 4485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.248690 (+ 1631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.248697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.248701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.248919 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.249656 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.249662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.249665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.250056 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:55.250275 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.251238 (+ 963us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.251546 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":314,"threads_started":1}
I20260807 22:42:55.253595 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.221892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.131:37247)
0807 22:42:55.223010 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.223092 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.226689 (+ 3597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.228162 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.228182 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.229259 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:55.230440 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.230491 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.232415 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.232425 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.235986 (+ 3561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.236023 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.237496 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.237502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.237723 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.238774 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:55.238815 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:55.242063 (+ 3248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:55.247306 (+ 5243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:55.247326 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:55.247356 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:55.248452 (+ 1096us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:55.249059 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:55.249066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:55.249069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:55.249487 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:55.250203 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:55.250223 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:55.250742 (+ 519us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.251133 (+ 391us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.251822 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":408,"threads_started":1}
I20260807 22:42:55.282718 22128 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:46859
I20260807 22:42:55.282796 22284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:46859 every 8 connection(s)
I20260807 22:42:55.284221 22128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:55.285558 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 22128
I20260807 22:42:55.285781 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:55.286409 20522 external_mini_cluster.cc:1428] Setting key 0608f0169f5335f71c676910096bd857
I20260807 22:42:55.287622 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.261963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.20.10.131:42147)
0807 22:42:55.271177 (+ 9214us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.271204 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.271439 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.271832 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.271838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.272735 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:55.273561 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.273580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.274863 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.274872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.276920 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.276946 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.278345 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.278350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.278495 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.279546 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:55.279581 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:55.280722 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:55.284237 (+ 3515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:55.284243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:55.284247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:55.284737 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:55.285229 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:55.285235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:55.285238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:55.285421 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:55.286217 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:55.286224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:55.286457 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.286608 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.287099 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9024,"thread_start_us":8854,"threads_started":1}
I20260807 22:42:55.288681 22285 heartbeater.cc:347] Connected to a master server at 127.20.10.190:40883
I20260807 22:42:55.288873 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.262692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:42147 (local address 127.20.10.190:40883)
0807 22:42:55.262897 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.262902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.271356 (+ 8454us) server_negotiation.cc:407] Connection header received
0807 22:42:55.271600 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.271604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.271681 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.271791 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:55.273763 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.274616 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.277130 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.277829 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.278018 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.280974 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:55.280979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:55.280982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:55.281055 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:55.284067 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.284885 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.284892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.284895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.285047 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.285603 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.285608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.285612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.286055 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:55.287968 (+ 1913us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.288105 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.288396 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 22:42:55.289057 22285 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:55.290195 22285 heartbeater.cc:511] Master 127.20.10.190:40883 requested a full tablet report, sending...
I20260807 22:42:55.292258 21746 ts_manager.cc:194] Registered new tserver with Master: 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859)
I20260807 22:42:55.293533 21746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:37247
I20260807 22:42:55.303905 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:55 dist-test-slave-csg5 krb5kdc[21697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:55 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:55.363490 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.345672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.20.10.190:40883)
0807 22:42:55.345885 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.345889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.345998 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:42:55.346207 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.346214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.346293 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.346399 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:55.348422 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.349459 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.351210 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.351757 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.351923 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.356619 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:55.356625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:55.356629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:55.356679 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:55.360096 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.361126 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.361133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.361136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.361404 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:55.361797 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:55.361801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:55.361804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:55.362143 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:55.362467 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.362791 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.363024 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:42:55.369061 21746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:55.377934 21746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:55.419256 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.393937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46345 (local address 127.0.0.1:60978)
0807 22:42:55.410472 (+ 16535us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.410492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.410622 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.411233 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.411237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.411261 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:55.411593 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.411601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.413327 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.413332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.415824 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.415841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.417824 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.417836 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.418119 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.418200 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.418404 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.418664 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16391,"mutex_wait_us":250,"thread_start_us":15070,"threads_started":1}
I20260807 22:42:55.419366 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.394416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.20.10.129:46345)
0807 22:42:55.408770 (+ 14354us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.408780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.410608 (+ 1828us) server_negotiation.cc:407] Connection header received
0807 22:42:55.410837 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.410843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.411014 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.411214 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:55.411781 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.413142 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.416030 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.417515 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.417690 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.417938 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.418811 (+ 873us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.419026 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14181,"thread_start_us":14022,"threads_started":1}
I20260807 22:42:55.419622 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.394373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.20.10.131:46859)
0807 22:42:55.408831 (+ 14458us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.408842 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.408888 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:42:55.409036 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.409047 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.409374 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.409621 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:55.410607 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.412678 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.416618 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.418177 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.418322 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.418558 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.418963 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.419167 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13178,"thread_start_us":13025,"threads_started":1}
I20260807 22:42:55.420662 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.393163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:46859 (local address 127.0.0.1:50062)
0807 22:42:55.406571 (+ 13408us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.406636 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.406915 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.409709 (+ 2794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.409718 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.409755 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:55.410374 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.410389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.412898 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.412905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.416385 (+ 3480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.416404 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.418454 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.418461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.418709 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.418746 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.419873 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.420364 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13022,"mutex_wait_us":309,"thread_start_us":12843,"threads_started":1}
I20260807 22:42:55.420714 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.393850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.20.10.130:35699)
0807 22:42:55.408336 (+ 14486us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.408351 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.409479 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 22:42:55.409663 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.409676 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.409993 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.410257 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:55.411435 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.413038 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.416664 (+ 3626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.417764 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.417925 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.418152 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.420167 (+ 2015us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.420416 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14280,"thread_start_us":14148,"threads_started":1}
I20260807 22:42:55.421502 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.393464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35699 (local address 127.0.0.1:44498)
0807 22:42:55.409312 (+ 15848us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.409335 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.409488 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.410518 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.410525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.410560 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:55.411154 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.411170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.413247 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.413253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.416446 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.416459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.418138 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.418145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.418344 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.418382 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.419762 (+ 1380us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.419917 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15686,"mutex_wait_us":146,"thread_start_us":16661,"threads_started":1}
I20260807 22:42:55.424803 22046 tablet_service.cc:1467] Processing CreateTablet for tablet 876dcd0d3bb447fb94288fd866f60890 (DEFAULT_TABLE table=test-table [id=b78e24ba6f5b4acebb3c42aa55978349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.424809 22204 tablet_service.cc:1467] Processing CreateTablet for tablet 876dcd0d3bb447fb94288fd866f60890 (DEFAULT_TABLE table=test-table [id=b78e24ba6f5b4acebb3c42aa55978349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.424821 21900 tablet_service.cc:1467] Processing CreateTablet for tablet 876dcd0d3bb447fb94288fd866f60890 (DEFAULT_TABLE table=test-table [id=b78e24ba6f5b4acebb3c42aa55978349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.426378 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876dcd0d3bb447fb94288fd866f60890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.426358 21900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876dcd0d3bb447fb94288fd866f60890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.426378 22046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876dcd0d3bb447fb94288fd866f60890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.440675 22306 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Bootstrap starting.
I20260807 22:42:55.443455 22307 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Bootstrap starting.
I20260807 22:42:55.444396 22309 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Bootstrap starting.
I20260807 22:42:55.445398 22306 tablet_bootstrap.cc:654] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.446944 22306 log.cc:824] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:55.448770 22307 tablet_bootstrap.cc:654] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.450079 22309 tablet_bootstrap.cc:654] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.450531 22307 log.cc:824] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:55.451396 22309 log.cc:824] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:55.454007 22309 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: No bootstrap required, opened a new log
I20260807 22:42:55.454013 22307 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: No bootstrap required, opened a new log
I20260807 22:42:55.454013 22306 tablet_bootstrap.cc:492] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: No bootstrap required, opened a new log
I20260807 22:42:55.454299 22309 ts_tablet_manager.cc:1397] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 22:42:55.454320 22306 ts_tablet_manager.cc:1397] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260807 22:42:55.454320 22307 ts_tablet_manager.cc:1397] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 22:42:55.469368 22309 raft_consensus.cc:348] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.469805 22306 raft_consensus.cc:348] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.470461 22309 raft_consensus.cc:374] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.470855 22306 raft_consensus.cc:374] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.471122 22306 raft_consensus.cc:729] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c1ea792d154736b29693b75acf0a97, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.471155 22309 raft_consensus.cc:729] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1597e0ad794496836832051f61bcb9, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.471725 22307 raft_consensus.cc:348] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.472544 22307 raft_consensus.cc:374] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.472361 22306 consensus_queue.cc:260] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.472786 22307 raft_consensus.cc:729] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.472465 22309 consensus_queue.cc:260] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.474249 22307 consensus_queue.cc:260] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.474882 22124 heartbeater.cc:503] Master 127.20.10.190:40883 was elected leader, sending a full tablet report...
I20260807 22:42:55.475805 22306 ts_tablet_manager.cc:1428] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Time spent starting tablet: real 0.021s user 0.017s sys 0.005s
I20260807 22:42:55.477314 22307 ts_tablet_manager.cc:1428] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Time spent starting tablet: real 0.023s user 0.014s sys 0.007s
I20260807 22:42:55.477654 22285 heartbeater.cc:503] Master 127.20.10.190:40883 was elected leader, sending a full tablet report...
I20260807 22:42:55.481478 22309 ts_tablet_manager.cc:1428] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Time spent starting tablet: real 0.027s user 0.016s sys 0.004s
W20260807 22:42:55.538120 22286 tablet.cc:2369] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:42:55.563841 21975 tablet.cc:2369] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:42:55.686924 22125 tablet.cc:2369] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:55.722899 22315 raft_consensus.cc:482] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:55.723486 22315 raft_consensus.cc:504] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.726841 22315 leader_election.cc:288] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859), f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699)
I20260807 22:42:55.740477 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.726713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48967 (local address 127.20.10.131:46859)
0807 22:42:55.727043 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.727051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.730302 (+ 3251us) server_negotiation.cc:407] Connection header received
0807 22:42:55.730648 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.730681 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.730779 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.730851 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:55.732237 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.733628 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.737537 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.738913 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.739089 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.739152 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.739886 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.740048 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 22:42:55.740813 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.727141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:33051 (local address 127.20.10.130:35699)
0807 22:42:55.727531 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.727539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.730362 (+ 2823us) server_negotiation.cc:407] Connection header received
0807 22:42:55.730758 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.730763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.730856 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.730957 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:55.733076 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.734486 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.737727 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.739244 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.739489 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.739587 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.740169 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.740333 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 22:42:55.740811 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.727335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:46859 (local address 127.20.10.129:48967)
0807 22:42:55.730144 (+ 2809us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.730164 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.730469 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.730894 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.730902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.730933 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:55.732052 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.732067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.733778 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.733784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.737276 (+ 3492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.737295 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.739403 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.739413 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.739690 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.739719 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.739809 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.739928 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2549,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":2062,"threads_started":1}
I20260807 22:42:55.741602 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.727394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35699 (local address 127.20.10.129:33051)
0807 22:42:55.730196 (+ 2802us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:55.730213 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:55.730568 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:55.731006 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:55.731013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:55.731042 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:55.732699 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.732715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.734776 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.734782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.737483 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:55.737504 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.739710 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.739718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:55.739948 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.739982 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:42:55.740083 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:42:55.740263 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2638,"mutex_wait_us":31,"spinlock_wait_cycles":178432,"thread_start_us":1947,"threads_started":1}
I20260807 22:42:55.742328 22226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876dcd0d3bb447fb94288fd866f60890" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9" is_pre_election: true
I20260807 22:42:55.743548 22226 raft_consensus.cc:2454] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:55.743508 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876dcd0d3bb447fb94288fd866f60890" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97" is_pre_election: true
I20260807 22:42:55.744683 22071 raft_consensus.cc:2454] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:55.744717 21841 leader_election.cc:302] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1597e0ad794496836832051f61bcb9, 842471cbc49f44199a5ff0d298552a64; no voters:
I20260807 22:42:55.745370 22315 raft_consensus.cc:2781] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:55.745662 22315 raft_consensus.cc:482] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:55.745847 22315 raft_consensus.cc:3037] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:55.749919 22315 raft_consensus.cc:504] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.751405 22315 leader_election.cc:288] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Requested vote from peers 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859), f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699)
I20260807 22:42:55.752044 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876dcd0d3bb447fb94288fd866f60890" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97"
I20260807 22:42:55.752110 22226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876dcd0d3bb447fb94288fd866f60890" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9"
I20260807 22:42:55.752370 22071 raft_consensus.cc:3037] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:55.752468 22226 raft_consensus.cc:3037] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:55.756628 22226 raft_consensus.cc:2454] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:55.757002 22071 raft_consensus.cc:2454] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:55.757377 21841 leader_election.cc:302] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1597e0ad794496836832051f61bcb9, 842471cbc49f44199a5ff0d298552a64; no voters:
I20260807 22:42:55.757946 22315 raft_consensus.cc:2781] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:55.758991 22315 raft_consensus.cc:686] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 1 LEADER]: Becoming Leader. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Running, Role: LEADER
I20260807 22:42:55.759696 22315 consensus_queue.cc:237] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } }
I20260807 22:42:55.767122 21746 catalog_manager.cc:5495] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842471cbc49f44199a5ff0d298552a64 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } health_report { overall_health: HEALTHY } } }
I20260807 22:42:55.801841 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:55.793176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.20.10.129:46345)
0807 22:42:55.793571 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:55.793578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:55.794142 (+ 564us) server_negotiation.cc:407] Connection header received
0807 22:42:55.794581 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:55.794585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:55.794776 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:55.794914 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:55.795470 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.796489 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.798318 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:55.799167 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:55.799275 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:55.800126 (+ 851us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:55.800959 (+ 833us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:55.801073 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:55.801243 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:42:55.801362 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:42:55.814445 22226 raft_consensus.cc:1261] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:55.816385 22071 raft_consensus.cc:1261] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:55.816103 22318 consensus_queue.cc:1035] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 22:42:55.817466 22315 consensus_queue.cc:1035] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:55.835731 22324 mvcc.cc:205] Tried to move back new op lower bound from 7316039990522118144 to 7316039990316994560. Current Snapshot: MvccSnapshot[applied={T|T < 7316039990522118144}]
I20260807 22:42:55.862365 21746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:55.862941 21746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:55.868583 21746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Sending DeleteTablet for 3 replicas of tablet 876dcd0d3bb447fb94288fd866f60890
I20260807 22:42:55.870540 22046 tablet_service.cc:1514] Processing DeleteTablet for tablet 876dcd0d3bb447fb94288fd866f60890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:55 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:44498
I20260807 22:42:55.871392 21900 tablet_service.cc:1514] Processing DeleteTablet for tablet 876dcd0d3bb447fb94288fd866f60890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:55 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:60978
I20260807 22:42:55.870522 22204 tablet_service.cc:1514] Processing DeleteTablet for tablet 876dcd0d3bb447fb94288fd866f60890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:55 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:50062
I20260807 22:42:55.872756 21746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:42:55.873270 22332 tablet_replica.cc:330] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: stopping tablet replica
I20260807 22:42:55.873934 22332 raft_consensus.cc:2229] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 22:42:55.873983 21746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:55.874737 22332 raft_consensus.cc:2258] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:55.876946 22333 tablet_replica.cc:330] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: stopping tablet replica
I20260807 22:42:55.877640 22333 raft_consensus.cc:2229] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:55.879756 22334 tablet_replica.cc:330] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: stopping tablet replica
I20260807 22:42:55.880353 22334 raft_consensus.cc:2229] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:55.881417 22334 raft_consensus.cc:2258] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:55.881589 22332 ts_tablet_manager.cc:1905] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:55.878756 22333 raft_consensus.cc:2258] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:55.884341 22334 ts_tablet_manager.cc:1905] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:55.885547 21900 tablet_service.cc:1467] Processing CreateTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd (DEFAULT_TABLE table=test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.885725 22204 tablet_service.cc:1467] Processing CreateTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd (DEFAULT_TABLE table=test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.886188 21900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9937c3979aab4f8a91131ed1c7f82ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.886399 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9937c3979aab4f8a91131ed1c7f82ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.888514 22046 tablet_service.cc:1467] Processing CreateTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd (DEFAULT_TABLE table=test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:55.889183 22046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9937c3979aab4f8a91131ed1c7f82ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:55.890396 22333 ts_tablet_manager.cc:1905] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:55.896382 22307 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Bootstrap starting.
I20260807 22:42:55.900590 22307 tablet_bootstrap.cc:654] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.902715 22306 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Bootstrap starting.
I20260807 22:42:55.906037 22306 tablet_bootstrap.cc:654] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.905966 22332 ts_tablet_manager.cc:1918] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:55.906344 22332 log.cc:1196] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/876dcd0d3bb447fb94288fd866f60890
I20260807 22:42:55.907439 22332 ts_tablet_manager.cc:1939] T 876dcd0d3bb447fb94288fd866f60890 P f9c1ea792d154736b29693b75acf0a97: Deleting consensus metadata
I20260807 22:42:55.905619 22333 ts_tablet_manager.cc:1918] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:55.908329 22333 log.cc:1196] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/876dcd0d3bb447fb94288fd866f60890
I20260807 22:42:55.909196 22333 ts_tablet_manager.cc:1939] T 876dcd0d3bb447fb94288fd866f60890 P 842471cbc49f44199a5ff0d298552a64: Deleting consensus metadata
I20260807 22:42:55.910518 21731 catalog_manager.cc:4841] TS f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699): tablet 876dcd0d3bb447fb94288fd866f60890 (table test-table [id=b78e24ba6f5b4acebb3c42aa55978349]) successfully deleted
I20260807 22:42:55.912938 22309 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Bootstrap starting.
I20260807 22:42:55.915410 21732 catalog_manager.cc:4841] TS 842471cbc49f44199a5ff0d298552a64 (127.20.10.129:46345): tablet 876dcd0d3bb447fb94288fd866f60890 (table test-table [id=b78e24ba6f5b4acebb3c42aa55978349]) successfully deleted
I20260807 22:42:55.916163 22309 tablet_bootstrap.cc:654] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:55.927263 22306 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: No bootstrap required, opened a new log
I20260807 22:42:55.927582 22307 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: No bootstrap required, opened a new log
I20260807 22:42:55.927786 22309 tablet_bootstrap.cc:492] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: No bootstrap required, opened a new log
I20260807 22:42:55.927938 22307 ts_tablet_manager.cc:1397] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.014s
I20260807 22:42:55.927649 22306 ts_tablet_manager.cc:1397] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.006s
I20260807 22:42:55.928082 22309 ts_tablet_manager.cc:1397] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.001s
I20260807 22:42:55.929450 22307 raft_consensus.cc:348] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.929548 22309 raft_consensus.cc:348] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.929600 22306 raft_consensus.cc:348] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.929804 22307 raft_consensus.cc:374] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.929771 22334 ts_tablet_manager.cc:1918] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:55.929921 22306 raft_consensus.cc:374] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.929921 22309 raft_consensus.cc:374] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:55.929944 22307 raft_consensus.cc:729] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.930051 22334 log.cc:1196] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/876dcd0d3bb447fb94288fd866f60890
I20260807 22:42:55.930104 22306 raft_consensus.cc:729] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c1ea792d154736b29693b75acf0a97, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.930261 22309 raft_consensus.cc:729] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1597e0ad794496836832051f61bcb9, State: Initialized, Role: FOLLOWER
I20260807 22:42:55.930473 22307 consensus_queue.cc:260] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.930562 22306 consensus_queue.cc:260] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.931510 22306 ts_tablet_manager.cc:1428] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 22:42:55.931407 22309 consensus_queue.cc:260] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:55.932374 22309 ts_tablet_manager.cc:1428] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 22:42:55.933470 22334 ts_tablet_manager.cc:1939] T 876dcd0d3bb447fb94288fd866f60890 P 4e1597e0ad794496836832051f61bcb9: Deleting consensus metadata
I20260807 22:42:55.936471 21730 catalog_manager.cc:4841] TS 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859): tablet 876dcd0d3bb447fb94288fd866f60890 (table test-table [id=b78e24ba6f5b4acebb3c42aa55978349]) successfully deleted
I20260807 22:42:55.938558 22307 ts_tablet_manager.cc:1428] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260807 22:42:56.110577 22315 raft_consensus.cc:482] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:56.110980 22315 raft_consensus.cc:504] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.112442 22315 leader_election.cc:288] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699), 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859)
I20260807 22:42:56.113152 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9937c3979aab4f8a91131ed1c7f82ffd" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97" is_pre_election: true
I20260807 22:42:56.113485 22071 raft_consensus.cc:2454] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:56.113255 22225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9937c3979aab4f8a91131ed1c7f82ffd" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9" is_pre_election: true
I20260807 22:42:56.113778 22225 raft_consensus.cc:2454] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:56.114316 21842 leader_election.cc:302] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842471cbc49f44199a5ff0d298552a64, f9c1ea792d154736b29693b75acf0a97; no voters:
I20260807 22:42:56.115149 22315 raft_consensus.cc:2781] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:56.115411 22315 raft_consensus.cc:482] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:56.115595 22315 raft_consensus.cc:3037] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.119295 22315 raft_consensus.cc:504] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.120299 22315 leader_election.cc:288] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Requested vote from peers f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699), 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859)
I20260807 22:42:56.121052 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9937c3979aab4f8a91131ed1c7f82ffd" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97"
I20260807 22:42:56.121124 22225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9937c3979aab4f8a91131ed1c7f82ffd" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9"
I20260807 22:42:56.121470 22225 raft_consensus.cc:3037] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.121407 22071 raft_consensus.cc:3037] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.126199 22225 raft_consensus.cc:2454] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:56.126199 22071 raft_consensus.cc:2454] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:56.126978 21842 leader_election.cc:302] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842471cbc49f44199a5ff0d298552a64, f9c1ea792d154736b29693b75acf0a97; no voters:
I20260807 22:42:56.127553 22315 raft_consensus.cc:2781] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:56.127815 22315 raft_consensus.cc:686] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 1 LEADER]: Becoming Leader. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Running, Role: LEADER
I20260807 22:42:56.128363 22315 consensus_queue.cc:237] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.132606 21744 catalog_manager.cc:5495] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842471cbc49f44199a5ff0d298552a64 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:42:56 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:56.196004 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.174871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:40883 (local address 127.0.0.1:50044)
0807 22:42:56.177386 (+ 2515us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.177405 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.177561 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.177970 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.177973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.178977 (+ 1004us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:56.179221 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.179232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.181552 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.181557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.182806 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.182822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.183989 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.183994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.184152 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.185091 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:56.185113 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:56.187879 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:56.192575 (+ 4696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.192590 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.192596 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.193196 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.193756 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.193761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.193764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.193991 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.194908 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:56.194916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:56.195112 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.195339 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.195646 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2369,"thread_start_us":2098,"threads_started":1}
I20260807 22:42:56.196395 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.174765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.20.10.190:40883)
0807 22:42:56.176344 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.176355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.177540 (+ 1185us) server_negotiation.cc:407] Connection header received
0807 22:42:56.177689 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.177696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.177790 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.177942 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.179441 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.181303 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.182976 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.183754 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.183906 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.188201 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.188210 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.188216 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.188260 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.192365 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.193392 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.193400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.193407 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.193586 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.194134 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.194140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.194144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.194652 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.194902 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.195482 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.195866 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":1295,"thread_start_us":900,"threads_started":1}
I20260807 22:42:56.203679 21744 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:50044:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:42:56.205438 21744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:56.218128 22204 tablet_service.cc:1467] Processing CreateTablet for tablet f1c833478a9c4d65a81beef703e75a61 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c38bcbb56604b35aca6eee6cea261a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:56.218865 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c833478a9c4d65a81beef703e75a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:56.219218 22046 tablet_service.cc:1467] Processing CreateTablet for tablet f1c833478a9c4d65a81beef703e75a61 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c38bcbb56604b35aca6eee6cea261a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:56.219389 21900 tablet_service.cc:1467] Processing CreateTablet for tablet f1c833478a9c4d65a81beef703e75a61 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c38bcbb56604b35aca6eee6cea261a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:56.219944 22046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c833478a9c4d65a81beef703e75a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:56.220170 21900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c833478a9c4d65a81beef703e75a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:56.231294 22309 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: Bootstrap starting.
I20260807 22:42:56.235667 22307 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: Bootstrap starting.
I20260807 22:42:56.235667 22306 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: Bootstrap starting.
I20260807 22:42:56.235865 22309 tablet_bootstrap.cc:654] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:56.240382 22306 tablet_bootstrap.cc:654] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:56.244618 22307 tablet_bootstrap.cc:654] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:56.249545 22306 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: No bootstrap required, opened a new log
I20260807 22:42:56.250128 22306 ts_tablet_manager.cc:1397] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260807 22:42:56.251791 22306 raft_consensus.cc:348] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.252345 22306 raft_consensus.cc:374] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:56.252555 22306 raft_consensus.cc:729] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c1ea792d154736b29693b75acf0a97, State: Initialized, Role: FOLLOWER
I20260807 22:42:56.253150 22306 consensus_queue.cc:260] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.254541 22313 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.255507 22313 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.257293 22306 ts_tablet_manager.cc:1428] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260807 22:42:56.261423 22309 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: No bootstrap required, opened a new log
I20260807 22:42:56.262029 22309 ts_tablet_manager.cc:1397] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: Time spent bootstrapping tablet: real 0.031s user 0.023s sys 0.004s
I20260807 22:42:56.264539 22309 raft_consensus.cc:348] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.265240 22309 raft_consensus.cc:374] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:56.265420 22309 raft_consensus.cc:729] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1597e0ad794496836832051f61bcb9, State: Initialized, Role: FOLLOWER
I20260807 22:42:56.265961 22309 consensus_queue.cc:260] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.266971 22314 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.267740 22314 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.267012 22309 ts_tablet_manager.cc:1428] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 22:42:56.272894 22307 tablet_bootstrap.cc:492] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: No bootstrap required, opened a new log
I20260807 22:42:56.273326 22307 ts_tablet_manager.cc:1397] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.010s
I20260807 22:42:56.274995 22307 raft_consensus.cc:348] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.275543 22307 raft_consensus.cc:374] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:56.275784 22307 raft_consensus.cc:729] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:56.276357 22307 consensus_queue.cc:260] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.277480 22307 ts_tablet_manager.cc:1428] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 22:42:56.277313 22315 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.278220 22315 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.364542 22315 raft_consensus.cc:482] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:56.365005 22315 raft_consensus.cc:504] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.366119 22315 leader_election.cc:288] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699), 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859)
I20260807 22:42:56.367067 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c833478a9c4d65a81beef703e75a61" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97" is_pre_election: true
I20260807 22:42:56.367383 22071 raft_consensus.cc:2454] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:56.367285 22225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c833478a9c4d65a81beef703e75a61" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9" is_pre_election: true
I20260807 22:42:56.367708 22225 raft_consensus.cc:2454] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 0.
I20260807 22:42:56.367996 21842 leader_election.cc:302] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842471cbc49f44199a5ff0d298552a64, f9c1ea792d154736b29693b75acf0a97; no voters:
I20260807 22:42:56.368427 22315 raft_consensus.cc:2781] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:56.368607 22315 raft_consensus.cc:482] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:56.368861 22315 raft_consensus.cc:3037] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.374568 22315 raft_consensus.cc:504] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.375919 22315 leader_election.cc:288] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Requested vote from peers f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699), 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859)
I20260807 22:42:56.376402 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c833478a9c4d65a81beef703e75a61" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c1ea792d154736b29693b75acf0a97"
I20260807 22:42:56.376719 22071 raft_consensus.cc:3037] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.376684 22225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c833478a9c4d65a81beef703e75a61" candidate_uuid: "842471cbc49f44199a5ff0d298552a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1597e0ad794496836832051f61bcb9"
I20260807 22:42:56.377310 22225 raft_consensus.cc:3037] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.379539 22071 raft_consensus.cc:2454] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:56.380205 21842 leader_election.cc:302] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842471cbc49f44199a5ff0d298552a64, f9c1ea792d154736b29693b75acf0a97; no voters:
I20260807 22:42:56.380713 22315 raft_consensus.cc:2781] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:56.381155 22315 raft_consensus.cc:686] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [term 1 LEADER]: Becoming Leader. State: Replica: 842471cbc49f44199a5ff0d298552a64, State: Running, Role: LEADER
I20260807 22:42:56.381847 22315 consensus_queue.cc:237] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } }
I20260807 22:42:56.382805 22225 raft_consensus.cc:2454] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842471cbc49f44199a5ff0d298552a64 in term 1.
I20260807 22:42:56.385608 22318 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: TxnStatusTablet state changed. Reason: New leader 842471cbc49f44199a5ff0d298552a64. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.386437 22318 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:56.388123 21744 catalog_manager.cc:5495] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842471cbc49f44199a5ff0d298552a64 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260807 22:42:56.388669 22352 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:42:56.389303 22352 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:42:56.446489 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.433948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.20.10.129:46345)
0807 22:42:56.436245 (+ 2297us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.436250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.436271 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:42:56.436347 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.436350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.436425 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.436524 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:56.437366 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.439139 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.442144 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.443346 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.443515 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.444928 (+ 1413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:56.445147 (+ 219us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:56.445245 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.445534 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.445642 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2169,"thread_start_us":1792,"threads_started":1}
I20260807 22:42:56.446988 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.434479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46345 (local address 127.0.0.1:60990)
0807 22:42:56.434889 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.434916 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.435162 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.436654 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.436660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.436676 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:56.436971 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.436989 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.439368 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.439375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.441836 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.441879 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.443714 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.443724 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.444357 (+ 633us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.444487 (+ 130us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:56.445306 (+ 819us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:56.445342 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.445476 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.445811 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:42:56.454890 22071 raft_consensus.cc:1261] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:56.454876 22225 raft_consensus.cc:1261] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:56.455833 22318 consensus_queue.cc:1035] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:56.456377 22315 consensus_queue.cc:1035] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:56.466315 22355 mvcc.cc:205] Tried to move back new op lower bound from 7316039993149530112 to 7316039992863563776. Current Snapshot: MvccSnapshot[applied={T|T < 7316039993149530112}]
I20260807 22:42:56.466602 22313 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: TxnStatusTablet state changed. Reason: New leader 842471cbc49f44199a5ff0d298552a64. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.467022 22313 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.469182 22313 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.469698 22313 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P f9c1ea792d154736b29693b75acf0a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.471115 22315 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.476023 22315 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:56.482092 22314 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: TxnStatusTablet state changed. Reason: New leader 842471cbc49f44199a5ff0d298552a64. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.482605 22356 mvcc.cc:205] Tried to move back new op lower bound from 7316039993149530112 to 7316039992863563776. Current Snapshot: MvccSnapshot[applied={T|T < 7316039993149530112 or (T in {7316039993149530112})}]
I20260807 22:42:56.482697 22314 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.487417 22315 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.488076 22315 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 842471cbc49f44199a5ff0d298552a64: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:56.495709 22314 tablet_replica.cc:439] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "842471cbc49f44199a5ff0d298552a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842471cbc49f44199a5ff0d298552a64" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46345 } } peers { permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 } } peers { permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 } } }
I20260807 22:42:56.496114 22314 tablet_replica.cc:442] T f1c833478a9c4d65a81beef703e75a61 P 4e1597e0ad794496836832051f61bcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:56.516448 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.501689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54055 (local address 127.20.10.129:46345)
0807 22:42:56.501949 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.501956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.504174 (+ 2218us) server_negotiation.cc:407] Connection header received
0807 22:42:56.504481 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.504487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.504621 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.504750 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:56.506576 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.508842 (+ 2266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.512607 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.514017 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.514247 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.515204 (+ 957us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:56.515373 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:56.515477 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.515688 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.515815 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:42:56.516973 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.501970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46345 (local address 127.20.10.129:54055)
0807 22:42:56.504020 (+ 2050us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.504046 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.504300 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.504856 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.504861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.504873 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:56.506062 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.506077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.509090 (+ 3013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.509095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.512313 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.512337 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.514437 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.514448 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.514872 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.514964 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:56.515499 (+ 535us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:56.515510 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.515584 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.515738 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1649,"thread_start_us":1205,"threads_started":1}
I20260807 22:42:56.531837 22071 raft_consensus.cc:1261] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:56.532265 22225 raft_consensus.cc:1261] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Refusing update from remote peer 842471cbc49f44199a5ff0d298552a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:56.532986 22366 consensus_queue.cc:1035] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9c1ea792d154736b29693b75acf0a97" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 22:42:56.533844 22315 consensus_queue.cc:1035] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1597e0ad794496836832051f61bcb9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:56.637185 21744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:56.637611 21744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:56.642601 21744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3abac79c86ec46bebbd433a16a5eaf73: Sending DeleteTablet for 3 replicas of tablet 9937c3979aab4f8a91131ed1c7f82ffd
I20260807 22:42:56.643922 21900 tablet_service.cc:1514] Processing DeleteTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:60978
I20260807 22:42:56.644284 22204 tablet_service.cc:1514] Processing DeleteTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:50062
I20260807 22:42:56.644320 22046 tablet_service.cc:1514] Processing DeleteTablet for tablet 9937c3979aab4f8a91131ed1c7f82ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:44498
I20260807 22:42:56.656251 22374 tablet_replica.cc:330] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: stopping tablet replica
I20260807 22:42:56.656898 22374 raft_consensus.cc:2229] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:56.657799 22374 raft_consensus.cc:2258] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:56.659653 22374 ts_tablet_manager.cc:1905] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:56.661922 22372 tablet_replica.cc:330] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: stopping tablet replica
I20260807 22:42:56.662536 22372 raft_consensus.cc:2229] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:56.663381 22373 tablet_replica.cc:330] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: stopping tablet replica
I20260807 22:42:56.663879 22373 raft_consensus.cc:2229] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:56.664516 22373 raft_consensus.cc:2258] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:56.666415 22373 ts_tablet_manager.cc:1905] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:56.667346 22372 raft_consensus.cc:2258] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:56.671114 22374 ts_tablet_manager.cc:1918] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:56.671620 22374 log.cc:1196] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/9937c3979aab4f8a91131ed1c7f82ffd
I20260807 22:42:56.671929 22372 ts_tablet_manager.cc:1905] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:56.673061 22374 ts_tablet_manager.cc:1939] T 9937c3979aab4f8a91131ed1c7f82ffd P f9c1ea792d154736b29693b75acf0a97: Deleting consensus metadata
I20260807 22:42:56.674916 21731 catalog_manager.cc:4841] TS f9c1ea792d154736b29693b75acf0a97 (127.20.10.130:35699): tablet 9937c3979aab4f8a91131ed1c7f82ffd (table test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]) successfully deleted
I20260807 22:42:56.679198 22373 ts_tablet_manager.cc:1918] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:56.679750 22373 log.cc:1196] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/9937c3979aab4f8a91131ed1c7f82ffd
I20260807 22:42:56.681051 22373 ts_tablet_manager.cc:1939] T 9937c3979aab4f8a91131ed1c7f82ffd P 4e1597e0ad794496836832051f61bcb9: Deleting consensus metadata
I20260807 22:42:56.682768 21730 catalog_manager.cc:4841] TS 4e1597e0ad794496836832051f61bcb9 (127.20.10.131:46859): tablet 9937c3979aab4f8a91131ed1c7f82ffd (table test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]) successfully deleted
I20260807 22:42:56.685716 22372 ts_tablet_manager.cc:1918] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:56.686014 22372 log.cc:1196] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/9937c3979aab4f8a91131ed1c7f82ffd
I20260807 22:42:56.687292 22372 ts_tablet_manager.cc:1939] T 9937c3979aab4f8a91131ed1c7f82ffd P 842471cbc49f44199a5ff0d298552a64: Deleting consensus metadata
I20260807 22:42:56.689158 21732 catalog_manager.cc:4841] TS 842471cbc49f44199a5ff0d298552a64 (127.20.10.129:46345): tablet 9937c3979aab4f8a91131ed1c7f82ffd (table test-table [id=9d4d0773cd9d4ff5b18b3779fa58a2bc]) successfully deleted
Aug 07 22:42:56 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:42:56.707746 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.655692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.20.10.129:46345)
0807 22:42:56.655994 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.656004 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.660743 (+ 4739us) server_negotiation.cc:407] Connection header received
0807 22:42:56.661124 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.661135 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.661276 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.661472 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.668209 (+ 6737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.670327 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.686625 (+ 16298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.688019 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.688160 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.697496 (+ 9336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.697520 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.697538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.697600 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.702776 (+ 5176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.704113 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.704123 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.704125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.704465 (+ 340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.705101 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.705105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.705108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.705755 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.706491 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.706835 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.707144 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
W20260807 22:42:56.709416 21968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32768
I20260807 22:42:56.746562 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.714403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.20.10.190:40883)
0807 22:42:56.718362 (+ 3959us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.718382 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.719062 (+ 680us) server_negotiation.cc:407] Connection header received
0807 22:42:56.719357 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.719363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.719444 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.719565 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.722013 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.723605 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.725924 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.726814 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.726981 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.733634 (+ 6653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.733647 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.733669 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.733833 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.741279 (+ 7446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.743078 (+ 1799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.743087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.743091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.743502 (+ 411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.744076 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.744082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.744085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.744699 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.745177 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.745494 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.745918 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3708,"thread_start_us":3544,"threads_started":1}
W20260807 22:42:56.747689 21744 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50060
I20260807 22:42:56.751945 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21824
I20260807 22:42:56.784231 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.753036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.20.10.129:46345)
0807 22:42:56.753449 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.753460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.761850 (+ 8390us) server_negotiation.cc:407] Connection header received
0807 22:42:56.762097 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.762103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.762194 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.762332 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.764922 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.766091 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.768566 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.769161 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.769333 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.773582 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.773589 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.773592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.773640 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.779496 (+ 5856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.780845 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.780858 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.780860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.781183 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.781791 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.781796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.781802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.782216 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.782985 (+ 769us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.783214 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.783656 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 22:42:56.784924 21968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32782
W20260807 22:42:56.785816 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21824: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:42:56.786160 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21824
I20260807 22:42:56.820037 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21978
Aug 07 22:42:56 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:42:56.843560 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.820961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.20.10.130:35699)
0807 22:42:56.822156 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.822162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.822181 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:42:56.822245 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.822249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.822320 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.822467 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.824619 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.825857 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.827999 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.828549 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.828672 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.835868 (+ 7196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.835891 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.835908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.835957 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.839670 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.840858 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.840868 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.840870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.841114 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.841763 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.841768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.841771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.842241 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.842532 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.842996 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.843209 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1039,"thread_start_us":853,"threads_started":1}
W20260807 22:42:56.844758 22118 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44504
W20260807 22:42:56.845800 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21978: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:42:56.845975 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21978
I20260807 22:42:56.871810 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22128
Aug 07 22:42:56 dist-test-slave-csg5 krb5kdc[21697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:42:56.890271 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.872742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.20.10.131:46859)
0807 22:42:56.873369 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.873374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.873394 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:42:56.873455 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.873458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.873524 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.873623 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.875365 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.876168 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.877684 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.878190 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.878308 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.883494 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.883513 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.883531 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.883583 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.887210 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.888089 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.888097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.888099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.888279 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.888676 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.888680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.888682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.889094 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.889397 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.889762 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.889971 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":317,"threads_started":1}
W20260807 22:42:56.891191 22279 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50076
W20260807 22:42:56.892158 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22128: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:42:56.892376 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 22128
I20260807 22:42:56.915268 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 21713
I20260807 22:42:56.916739 21792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:42:57.050716 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 21713
2026-08-07T22:42:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5671 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:57 dist-test-slave-csg5 krb5kdc[22386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:57 dist-test-slave-csg5 krb5kdc[22386](info): set up 2 sockets
Aug 07 22:42:57 dist-test-slave-csg5 krb5kdc[22386](info): commencing operation
krb5kdc: starting...
W20260807 22:42:58.275450 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:42:58 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:42:58.532116 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35507
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:58.893208 22401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:58.893599 22401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:58.893785 22401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:58.901540 22401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:58.901731 22401 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:42:58.901844 22401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:58.901960 22401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:58.902053 22401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:58.902148 22401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:58.913478 22401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35507
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:58.915050 22401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:58.916535 22401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:58.929880 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.929909 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.930234 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.930786 22401 server_base.cc:1034] running on GCE node
I20260807 22:42:58.931670 22401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:58.933353 22401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:58.934705 22401 hybrid_clock.cc:648] HybridClock initialized: now 1786142578934563 us; error 61 us; skew 500 ppm
I20260807 22:42:58.935129 22401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:58 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:58.939102 22401 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:42:58.940945 22401 webserver.cc:460] Webserver started at http://127.20.10.190:43113/ using document root <none> and password file <none>
I20260807 22:42:58.941766 22401 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:58.941906 22401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:58.942257 22401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:58.945600 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6369e199f7a342439ada581f94bd5b82"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-csg5"
server_key: "78356bf6b19fb812bfcb22d52c883ea5"
server_key_iv: "85094fcca49227717241dc372651b2fe"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.946519 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6369e199f7a342439ada581f94bd5b82"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-csg5"
server_key: "78356bf6b19fb812bfcb22d52c883ea5"
server_key_iv: "85094fcca49227717241dc372651b2fe"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.952581 22401 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:42:58.956923 22416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.958935 22401 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:42:58.959187 22401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6369e199f7a342439ada581f94bd5b82"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-csg5"
server_key: "78356bf6b19fb812bfcb22d52c883ea5"
server_key_iv: "85094fcca49227717241dc372651b2fe"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.959491 22401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:58.976222 22401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:58.981911 22401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:58.982203 22401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:59.007198 22401 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:35507
I20260807 22:42:59.007397 22485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:35507 every 8 connection(s)
I20260807 22:42:59.008893 22401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:59.013288 22486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:59.017972 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 22401
I20260807 22:42:59.018251 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:42:59.019126 20522 external_mini_cluster.cc:1428] Setting key 521f41dc9bb5923895e108ff06a2148f
I20260807 22:42:59.026748 22486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Bootstrap starting.
I20260807 22:42:59.030954 22486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:59.032253 22486 log.cc:824] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:59.035410 22486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: No bootstrap required, opened a new log
Aug 07 22:42:59 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:59.044166 22486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } }
I20260807 22:42:59.044870 22486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:59.045030 22486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6369e199f7a342439ada581f94bd5b82, State: Initialized, Role: FOLLOWER
I20260807 22:42:59.045862 22486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } }
I20260807 22:42:59.046279 22486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:59.046465 22486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:59.046749 22486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:59.049263 22486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } }
I20260807 22:42:59.049861 22486 leader_election.cc:302] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6369e199f7a342439ada581f94bd5b82; no voters:
I20260807 22:42:59.050522 22486 leader_election.cc:288] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:59.050782 22492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:59.051973 22492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [term 1 LEADER]: Becoming Leader. State: Replica: 6369e199f7a342439ada581f94bd5b82, State: Running, Role: LEADER
I20260807 22:42:59.052853 22486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:59.052722 22492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } }
I20260807 22:42:59.053412 22490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:59.021706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.20.10.190:35507)
0807 22:42:59.022705 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:59.022726 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:59.022781 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:42:59.025062 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:59.025100 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:59.025545 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:59.025906 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:59.028021 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.029610 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.031524 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.032336 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.032588 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:59.042442 (+ 9854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:59.042459 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:59.042480 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:59.042538 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:59.047357 (+ 4819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.048228 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.048253 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.048260 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.048458 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.048891 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.048897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.048900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.049549 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:59.049889 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:59.051168 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0807 22:42:59.051832 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":477,"threads_started":1}
I20260807 22:42:59.057138 22493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6369e199f7a342439ada581f94bd5b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } } }
I20260807 22:42:59.057619 22493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:59.058223 22494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6369e199f7a342439ada581f94bd5b82. Latest consensus state: current_term: 1 leader_uuid: "6369e199f7a342439ada581f94bd5b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6369e199f7a342439ada581f94bd5b82" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35507 } } }
I20260807 22:42:59.058544 22494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:59.058568 22501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:59.065568 22501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:59.078439 22501 catalog_manager.cc:1347] Generated new cluster ID: 87b0ba61cbc9470593e2914417d011aa
I20260807 22:42:59.078583 22501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:59.087368 22501 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:59.088768 22501 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:59.098568 22501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Generated new TSK 0
I20260807 22:42:59.099650 22501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:42:59.159571 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35507
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:59.542547 22514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:59.542949 22514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:59.543138 22514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:59.551342 22514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:59.551503 22514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:59.551667 22514 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:42:59.551765 22514 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:59.551894 22514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:42:59.563387 22514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:42:59.564888 22514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:59.566349 22514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:59.580979 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:59.580945 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:59.580912 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:59.581804 22514 server_base.cc:1034] running on GCE node
I20260807 22:42:59.582738 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:59.583948 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:59.585220 22514 hybrid_clock.cc:648] HybridClock initialized: now 1786142579585156 us; error 55 us; skew 500 ppm
I20260807 22:42:59.585666 22514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:42:59 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142579, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:59.589570 22514 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:42:59.591599 22514 webserver.cc:460] Webserver started at http://127.20.10.129:35781/ using document root <none> and password file <none>
I20260807 22:42:59.592396 22514 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:59.592525 22514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:59.592849 22514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:59.596410 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
format_stamp: "Formatted at 2026-08-07 22:42:59 on dist-test-slave-csg5"
server_key: "ca7aa657c77fec13148866539448e56c"
server_key_iv: "4e6dcf1e82e91375f0b4966836dc9c55"
server_key_version: "encryptionkey@0"
I20260807 22:42:59.597307 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
format_stamp: "Formatted at 2026-08-07 22:42:59 on dist-test-slave-csg5"
server_key: "ca7aa657c77fec13148866539448e56c"
server_key_iv: "4e6dcf1e82e91375f0b4966836dc9c55"
server_key_version: "encryptionkey@0"
I20260807 22:42:59.604501 22514 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 22:42:59.609253 22529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:59.611099 22514 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:42:59.611336 22514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
format_stamp: "Formatted at 2026-08-07 22:42:59 on dist-test-slave-csg5"
server_key: "ca7aa657c77fec13148866539448e56c"
server_key_iv: "4e6dcf1e82e91375f0b4966836dc9c55"
server_key_version: "encryptionkey@0"
I20260807 22:42:59.611582 22514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:59.643008 22514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:59.648319 22514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:59.648651 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:59.652029 22514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:59.652172 22514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:59.652356 22514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:59.652454 22514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:59 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142579, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:42:59.682938 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:59.653433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.129:48477)
0807 22:42:59.654581 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:59.654632 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:59.656867 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:59.657737 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:59.657750 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:59.658647 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:59.659826 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:59.659855 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.661599 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.661603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:59.664212 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:59.664228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.665861 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.665868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:59.666083 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:59.666938 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:59.666965 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:59.670908 (+ 3943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:59.675749 (+ 4841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:59.675765 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:59.675790 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:59.676715 (+ 925us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:59.677367 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:59.677377 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:59.677384 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:59.677960 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:59.678951 (+ 991us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:59.678965 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:59.679589 (+ 624us) client_negotiation.cc:769] Sending connection context
0807 22:42:59.680071 (+ 482us) client_negotiation.cc:240] Negotiation successful
0807 22:42:59.680943 (+ 872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":452,"threads_started":1}
I20260807 22:42:59.684092 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:59.652582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48477 (local address 127.20.10.190:35507)
0807 22:42:59.653593 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:59.653598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:59.654830 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 22:42:59.657208 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:59.657214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:59.657315 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:59.657444 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:59.660054 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.661323 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.664418 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.665470 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.665679 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:59.671260 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:59.671268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:59.671273 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:59.671326 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:59.675529 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.676927 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.676936 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.676942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.677166 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.678203 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.678211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.678215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.678710 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:59.678996 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:59.683029 (+ 4033us) server_negotiation.cc:299] Negotiation successful
0807 22:42:59.683446 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":725,"threads_started":1}
I20260807 22:42:59.711797 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:59.693484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:49215 (local address 127.20.10.190:35507)
0807 22:42:59.693760 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:59.693770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:59.693927 (+ 157us) server_negotiation.cc:407] Connection header received
0807 22:42:59.694165 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:59.694175 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:59.694294 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:59.694454 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:59.696216 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.697793 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.700359 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.701362 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.701557 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:59.704470 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:59.704478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:59.704488 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:59.704526 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:59.708554 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.709444 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.709450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.709454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.709610 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.710044 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.710050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.710054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.710387 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:59.710585 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:59.711113 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 22:42:59.711381 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:42:59.711935 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:59.692862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.129:49215)
0807 22:42:59.693753 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:59.693792 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:59.694013 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:59.694478 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:59.694481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:59.695293 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:59.695938 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:59.695963 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.698049 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.698058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:59.700115 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:59.700147 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:59.701613 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:59.701618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:59.701752 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:59.702593 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:59.702614 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:59.704137 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:59.708776 (+ 4639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:59.708785 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:59.708788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:59.709282 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:59.709749 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:59.709755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:59.709759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:59.709912 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:59.710520 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:59.710526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:59.710810 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 22:42:59.710982 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 22:42:59.711409 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":510,"threads_started":1}
I20260807 22:42:59.722014 22514 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:45019
I20260807 22:42:59.722107 22660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:45019 every 8 connection(s)
I20260807 22:42:59.723461 22514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:59.725811 22662 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35507
I20260807 22:42:59.726065 22662 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:59.726167 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 22514
I20260807 22:42:59.726398 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:59.726938 22662 heartbeater.cc:511] Master 127.20.10.190:35507 requested a full tablet report, sending...
I20260807 22:42:59.727236 20522 external_mini_cluster.cc:1428] Setting key e0508c7ded55c6393ea24c79be62cf46
I20260807 22:42:59.729431 22435 ts_manager.cc:194] Registered new tserver with Master: 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019)
I20260807 22:42:59.732111 22435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:48477
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:42:59.783625 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35507
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:00.158890 22666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:00.159230 22666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:00.159490 22666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:00.167747 22666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:00.167881 22666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:00.168025 22666 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:43:00.168089 22666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:00.168190 22666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:00.178782 22666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:00.180214 22666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:00.181694 22666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:00.198139 22672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.198241 22674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.198215 22671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.199227 22666 server_base.cc:1034] running on GCE node
I20260807 22:43:00.199967 22666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:00.201004 22666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:00.202270 22666 hybrid_clock.cc:648] HybridClock initialized: now 1786142580202194 us; error 53 us; skew 500 ppm
I20260807 22:43:00.202780 22666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:00 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:00.206522 22666 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:00.208901 22666 webserver.cc:460] Webserver started at http://127.20.10.130:42471/ using document root <none> and password file <none>
I20260807 22:43:00.209755 22666 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:00.209937 22666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:00.210337 22666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:00.214217 22666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "c7e76c67f2634dd6987cfb3119bac047"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "7dcb21a826cfc37300d6b1e8e1fb8fb6"
server_key_iv: "bf80cccf276d187d5481523b08bdaf9c"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.215240 22666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "c7e76c67f2634dd6987cfb3119bac047"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "7dcb21a826cfc37300d6b1e8e1fb8fb6"
server_key_iv: "bf80cccf276d187d5481523b08bdaf9c"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.222131 22666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:43:00.226940 22681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.228826 22666 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:00.229071 22666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "c7e76c67f2634dd6987cfb3119bac047"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "7dcb21a826cfc37300d6b1e8e1fb8fb6"
server_key_iv: "bf80cccf276d187d5481523b08bdaf9c"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.229360 22666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:00.250880 22666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:00.256781 22666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:00.257153 22666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:00.261149 22666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:00.261317 22666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.261498 22666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:00.261606 22666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:00 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:00.286405 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.261123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:60835 (local address 127.20.10.190:35507)
0807 22:43:00.261820 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.261827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.263093 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 22:43:00.265582 (+ 2489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.265592 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.265724 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.265891 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.268066 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.269297 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.272199 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.272666 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.272817 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.278205 (+ 5388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.278210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.278231 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.278287 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.281990 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.283157 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.283174 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.283177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.283406 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.284261 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.284271 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.284275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.284662 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.284847 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.285718 (+ 871us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.285984 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":347,"threads_started":1}
I20260807 22:43:00.287751 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.261852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.130:60835)
0807 22:43:00.262880 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.262942 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.265252 (+ 2310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.266134 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.266142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.266797 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.267777 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.267811 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.269483 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.269487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.271992 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.272019 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.272972 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.272981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.273237 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.274158 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.274197 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.277892 (+ 3695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.282179 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.282190 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.282215 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.282994 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.283577 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.283587 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.283595 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.284074 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.284816 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.284827 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.285272 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.285594 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.286210 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":438,"threads_started":1}
I20260807 22:43:00.315078 22666 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:41963
I20260807 22:43:00.315078 22809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:41963 every 8 connection(s)
I20260807 22:43:00.316360 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.296756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:53491 (local address 127.20.10.190:35507)
0807 22:43:00.297017 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.297027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.297753 (+ 726us) server_negotiation.cc:407] Connection header received
0807 22:43:00.298043 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.298051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.298144 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.298255 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.299752 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.300807 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.302853 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.303799 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.303976 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.307950 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.307958 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.307964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.308017 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.312671 (+ 4654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.313741 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.313750 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.313754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.313924 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.314371 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.314377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.314381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.314911 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.315091 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.315653 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.315922 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:43:00.316455 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.296719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.130:53491)
0807 22:43:00.297576 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.297605 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.297854 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.298298 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.298303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.298979 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.299592 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.299605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.300944 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.300950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.302624 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.302647 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.304338 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.304346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.304541 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.305440 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.305473 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.307468 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.312882 (+ 5414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.312888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.312892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.313519 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.314070 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.314073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.314076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.314235 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.315048 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.315053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.315320 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.315503 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.315905 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":396,"threads_started":1}
I20260807 22:43:00.316931 22666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:00.319679 22810 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35507
I20260807 22:43:00.319967 22810 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:00.320008 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 22666
I20260807 22:43:00.320283 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:00.320794 20522 external_mini_cluster.cc:1428] Setting key 57e10b820ce5e9592afc9bc2cbd1a59c
I20260807 22:43:00.321000 22810 heartbeater.cc:511] Master 127.20.10.190:35507 requested a full tablet report, sending...
I20260807 22:43:00.323172 22434 ts_manager.cc:194] Registered new tserver with Master: c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:00.324100 22434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:60835
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:00.379123 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35507
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:00.735906 22662 heartbeater.cc:503] Master 127.20.10.190:35507 was elected leader, sending a full tablet report...
W20260807 22:43:00.831389 22815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:00.831744 22815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:00.831918 22815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:00.839939 22815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:00.840137 22815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:00.840283 22815 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:43:00.840372 22815 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:00.840502 22815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:00.851184 22815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:00.852658 22815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:00.854229 22815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:00.870587 22820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.870810 22823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.870610 22821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.871532 22815 server_base.cc:1034] running on GCE node
I20260807 22:43:00.872387 22815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:00.873438 22815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:00.874795 22815 hybrid_clock.cc:648] HybridClock initialized: now 1786142580874727 us; error 51 us; skew 500 ppm
I20260807 22:43:00.875134 22815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:00 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:00.878859 22815 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:00.880457 22815 webserver.cc:460] Webserver started at http://127.20.10.131:34421/ using document root <none> and password file <none>
I20260807 22:43:00.881196 22815 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:00.881352 22815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:00.881690 22815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:00.885110 22815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "3a29098db2a94afca833c05c5c997598"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "dd6d2df6168c52cb55f2539650c4cd2d"
server_key_iv: "bf5d49007e83ad7c95cedb359ecfb509"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.886018 22815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "3a29098db2a94afca833c05c5c997598"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "dd6d2df6168c52cb55f2539650c4cd2d"
server_key_iv: "bf5d49007e83ad7c95cedb359ecfb509"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.892499 22815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 22:43:00.897094 22831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.899184 22815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:00.899456 22815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "3a29098db2a94afca833c05c5c997598"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-csg5"
server_key: "dd6d2df6168c52cb55f2539650c4cd2d"
server_key_iv: "bf5d49007e83ad7c95cedb359ecfb509"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.899747 22815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:00.925180 22815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:00.930063 22815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:00.930402 22815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:00.933425 22815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:00.933666 22815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.933840 22815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:00.933940 22815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:00 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:00.965785 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.934528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:56287 (local address 127.20.10.190:35507)
0807 22:43:00.935179 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.935186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.936719 (+ 1533us) server_negotiation.cc:407] Connection header received
0807 22:43:00.940191 (+ 3472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.940197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.940310 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.940458 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.943656 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.944717 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.948352 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.949012 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.949179 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.956326 (+ 7147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.956333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.956336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.956752 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.960471 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.961822 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.961832 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.961838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.962076 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.962916 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.962924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.962931 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.963454 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.963642 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.964673 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.965107 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":301,"threads_started":1}
I20260807 22:43:00.966401 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.935479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.131:56287)
0807 22:43:00.936480 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.936552 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.939699 (+ 3147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.940854 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.940875 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.942080 (+ 1205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.943398 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.943434 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.944946 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.944954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.948105 (+ 3151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.948141 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.949630 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.949647 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.950309 (+ 662us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.951713 (+ 1404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.951772 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.955962 (+ 4190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.960687 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.960700 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.960733 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.961576 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.962261 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.962268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.962274 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.962737 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.963642 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.963660 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.964142 (+ 482us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.964423 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.964904 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":396,"threads_started":1}
I20260807 22:43:00.992910 22815 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43213
I20260807 22:43:00.992964 22960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43213 every 8 connection(s)
I20260807 22:43:00.994767 22815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:00.995397 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.973476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:50569 (local address 127.20.10.190:35507)
0807 22:43:00.973754 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.973764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.975155 (+ 1391us) server_negotiation.cc:407] Connection header received
0807 22:43:00.975499 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.975505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.975605 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.975760 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.977698 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.979028 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.981968 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.982507 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.982710 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.986493 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.986504 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.986513 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.986582 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.991127 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.992804 (+ 1677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.992812 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.992816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.993021 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.993530 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.993535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.993539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.993933 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.994093 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.994822 (+ 729us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.995043 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:43:00.995683 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.973440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.20.10.131:50569)
0807 22:43:00.974979 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.975009 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.975269 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.975921 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.975928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.976840 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.977434 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.977456 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.979229 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.979238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.981760 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.981779 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.982824 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.982840 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.983108 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.984084 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.984119 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.986064 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.991350 (+ 5286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.991360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.991368 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.992596 (+ 1228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.993192 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.993198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.993202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.993380 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.994076 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.994084 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.994349 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.994550 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.995013 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"thread_start_us":771,"threads_started":1}
I20260807 22:43:00.996840 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 22815
I20260807 22:43:00.997040 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:00.997707 20522 external_mini_cluster.cc:1428] Setting key f74707dc3ca678e17fd879bc7aeee707
I20260807 22:43:00.998344 22961 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35507
I20260807 22:43:00.998641 22961 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:00.999909 22961 heartbeater.cc:511] Master 127.20.10.190:35507 requested a full tablet report, sending...
I20260807 22:43:01.002934 22434 ts_manager.cc:194] Registered new tserver with Master: 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:01.003825 22434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:56287
I20260807 22:43:01.014451 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:43:01 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:43:01 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:01.079149 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.061278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.20.10.190:35507)
0807 22:43:01.061624 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.061633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.061837 (+ 204us) server_negotiation.cc:407] Connection header received
0807 22:43:01.061997 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.062003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.062143 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.062285 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.064269 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.065215 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.066865 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.067334 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.067449 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.072199 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.072207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.072212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:01.072258 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.075758 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.076686 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.076693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.076697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.076843 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.077453 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.077462 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.077469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.077858 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.078094 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.078471 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.078741 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260807 22:43:01.084681 22434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:01.092413 22434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:01.119234 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.109051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.20.10.129:45019)
0807 22:43:01.109928 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.109941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.111134 (+ 1193us) server_negotiation.cc:407] Connection header received
0807 22:43:01.111231 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.111237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.111435 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.111716 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.112286 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.113496 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.115975 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.117685 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.117844 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.118073 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.118619 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.118884 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":443,"threads_started":1}
I20260807 22:43:01.119884 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.108492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:45019 (local address 127.0.0.1:59450)
0807 22:43:01.110701 (+ 2209us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.110743 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.110956 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.111720 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.111727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.111758 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.112124 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.112135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.113636 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.113644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.115695 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.115709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.117964 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.117977 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.118243 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.118305 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.118573 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.119420 (+ 847us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1912,"mutex_wait_us":46,"thread_start_us":2810,"threads_started":1}
I20260807 22:43:01.122141 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.109886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43213 (local address 127.0.0.1:57338)
0807 22:43:01.111618 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.111640 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.111899 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.113055 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.113064 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.113107 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.113739 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.113760 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.116189 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.116198 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.118571 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.118587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.121289 (+ 2702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.121297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.121465 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.121501 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.121619 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.121775 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1567,"thread_start_us":364,"threads_started":1}
I20260807 22:43:01.122251 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.108899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.20.10.131:43213)
0807 22:43:01.109719 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.109730 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.111758 (+ 2028us) server_negotiation.cc:407] Connection header received
0807 22:43:01.112141 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.112149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.112415 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.112616 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.114073 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.115985 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.118847 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.120531 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.120665 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.121030 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.121707 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.121902 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":398,"threads_started":1}
I20260807 22:43:01.124750 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.112586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.0.0.1:42886)
0807 22:43:01.115180 (+ 2594us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.115202 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.115330 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.116332 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.116338 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.116374 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.116852 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.116865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.118735 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.118740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.122168 (+ 3428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.122186 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.123871 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.123878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.124061 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.124102 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.124221 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.124377 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2447,"thread_start_us":2272,"threads_started":1}
I20260807 22:43:01.124769 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.112137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.20.10.130:41963)
0807 22:43:01.112780 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.112789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.115460 (+ 2671us) server_negotiation.cc:407] Connection header received
0807 22:43:01.115568 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.115578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.115952 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.116152 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.117090 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.118503 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.122425 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.123647 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.123819 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.124057 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.124275 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.124450 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":622,"threads_started":1}
I20260807 22:43:01.124318 22582 tablet_service.cc:1467] Processing CreateTablet for tablet 509541329d404604ad6bb276e16e4176 (DEFAULT_TABLE table=test-table [id=ec1db060bd584fb3835ff4be46c15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.126029 22582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509541329d404604ad6bb276e16e4176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.126482 22885 tablet_service.cc:1467] Processing CreateTablet for tablet 509541329d404604ad6bb276e16e4176 (DEFAULT_TABLE table=test-table [id=ec1db060bd584fb3835ff4be46c15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.128156 22885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509541329d404604ad6bb276e16e4176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.127904 22735 tablet_service.cc:1467] Processing CreateTablet for tablet 509541329d404604ad6bb276e16e4176 (DEFAULT_TABLE table=test-table [id=ec1db060bd584fb3835ff4be46c15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.129386 22735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509541329d404604ad6bb276e16e4176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.145047 22981 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Bootstrap starting.
I20260807 22:43:01.148729 22981 tablet_bootstrap.cc:654] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.149272 22982 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Bootstrap starting.
I20260807 22:43:01.150051 22981 log.cc:824] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.152454 22983 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Bootstrap starting.
I20260807 22:43:01.154006 22982 tablet_bootstrap.cc:654] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.156843 22982 log.cc:824] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.157194 22983 tablet_bootstrap.cc:654] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.158602 22983 log.cc:824] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.161342 22982 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: No bootstrap required, opened a new log
I20260807 22:43:01.161651 22982 ts_tablet_manager.cc:1397] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260807 22:43:01.162055 22983 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: No bootstrap required, opened a new log
I20260807 22:43:01.162344 22983 ts_tablet_manager.cc:1397] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260807 22:43:01.164955 22981 tablet_bootstrap.cc:492] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: No bootstrap required, opened a new log
I20260807 22:43:01.165318 22981 ts_tablet_manager.cc:1397] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.014s
I20260807 22:43:01.169894 22982 raft_consensus.cc:348] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.170369 22982 raft_consensus.cc:374] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.170475 22982 raft_consensus.cc:729] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.171233 22982 consensus_queue.cc:260] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.174918 22982 ts_tablet_manager.cc:1428] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260807 22:43:01.175027 22983 raft_consensus.cc:348] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.175529 22983 raft_consensus.cc:374] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.175638 22983 raft_consensus.cc:729] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.176360 22983 consensus_queue.cc:260] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.177651 22961 heartbeater.cc:503] Master 127.20.10.190:35507 was elected leader, sending a full tablet report...
I20260807 22:43:01.178465 22983 ts_tablet_manager.cc:1428] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260807 22:43:01.178689 22981 raft_consensus.cc:348] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.179280 22981 raft_consensus.cc:374] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.179426 22981 raft_consensus.cc:729] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.180167 22981 consensus_queue.cc:260] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.181483 22810 heartbeater.cc:503] Master 127.20.10.190:35507 was elected leader, sending a full tablet report...
I20260807 22:43:01.182859 22981 ts_tablet_manager.cc:1428] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Time spent starting tablet: real 0.017s user 0.012s sys 0.006s
I20260807 22:43:01.195817 22987 raft_consensus.cc:482] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:01.196173 22987 raft_consensus.cc:504] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.198004 22987 leader_election.cc:288] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:01.207652 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.198473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:44523 (local address 127.20.10.130:41963)
0807 22:43:01.198756 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.198765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.199648 (+ 883us) server_negotiation.cc:407] Connection header received
0807 22:43:01.199855 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.199860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.199919 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.200014 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.200636 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.201440 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.204643 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.206040 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.206262 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.206346 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.207166 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.207341 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:43:01.207649 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.198453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:58633 (local address 127.20.10.131:43213)
0807 22:43:01.198739 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.198748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.199519 (+ 771us) server_negotiation.cc:407] Connection header received
0807 22:43:01.199753 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.199757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.199824 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.199923 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.200558 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.201326 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.204621 (+ 3295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.206075 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.206271 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.206346 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.207166 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.207341 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:43:01.207923 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.198349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.20.10.129:44523)
0807 22:43:01.199528 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.199547 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.199710 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.200030 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.200035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.200069 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.200477 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.200485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.201857 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.201867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.204456 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.204472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.206511 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.206520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.206908 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.206980 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.207107 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.207322 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":91,"thread_start_us":737,"threads_started":1}
I20260807 22:43:01.208353 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.198265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43213 (local address 127.20.10.129:58633)
0807 22:43:01.199395 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.199421 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.199601 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.199940 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.199944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.199968 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.200408 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.200418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.201460 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.201467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.204344 (+ 2877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.204368 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.206528 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.206534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.206923 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.206983 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.207107 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.207322 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":111,"thread_start_us":702,"threads_started":1}
I20260807 22:43:01.209122 22761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509541329d404604ad6bb276e16e4176" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047" is_pre_election: true
I20260807 22:43:01.209271 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509541329d404604ad6bb276e16e4176" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598" is_pre_election: true
I20260807 22:43:01.209807 22761 raft_consensus.cc:2454] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 0.
I20260807 22:43:01.209868 22907 raft_consensus.cc:2454] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 0.
I20260807 22:43:01.210800 22530 leader_election.cc:302] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ba98cb2c87d421a9c63c9b8fb43d75f, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:01.211406 22987 raft_consensus.cc:2781] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:01.211591 22987 raft_consensus.cc:482] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:01.211719 22987 raft_consensus.cc:3037] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.214107 22987 raft_consensus.cc:504] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.214947 22987 leader_election.cc:288] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 election: Requested vote from peers 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:01.215643 22761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509541329d404604ad6bb276e16e4176" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047"
I20260807 22:43:01.215662 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509541329d404604ad6bb276e16e4176" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598"
I20260807 22:43:01.215900 22761 raft_consensus.cc:3037] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.215888 22907 raft_consensus.cc:3037] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.218467 22907 raft_consensus.cc:2454] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 1.
I20260807 22:43:01.218480 22761 raft_consensus.cc:2454] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 1.
I20260807 22:43:01.219007 22533 leader_election.cc:302] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, 3ba98cb2c87d421a9c63c9b8fb43d75f; no voters:
I20260807 22:43:01.219401 22987 raft_consensus.cc:2781] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:01.219887 22987 raft_consensus.cc:686] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 LEADER]: Becoming Leader. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Running, Role: LEADER
I20260807 22:43:01.220402 22987 consensus_queue.cc:237] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } }
I20260807 22:43:01.225675 22434 catalog_manager.cc:5495] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129). New cstate: current_term: 1 leader_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } health_report { overall_health: UNKNOWN } } }
W20260807 22:43:01.226447 22663 tablet.cc:2369] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:01.252835 22962 tablet.cc:2369] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:43:01 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:01.293540 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.277545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.20.10.190:35507)
0807 22:43:01.277774 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.277779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.278399 (+ 620us) server_negotiation.cc:407] Connection header received
0807 22:43:01.278563 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.278568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.278635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.278785 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.280089 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.281037 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.282427 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.282975 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.283098 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.286954 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.286974 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.286977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:01.287021 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.290110 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.291246 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.291272 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.291276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.291437 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.291840 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.291846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.291850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.292186 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.292350 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.292792 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.293180 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":91}
I20260807 22:43:01.293547 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.277582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35507 (local address 127.0.0.1:52932)
0807 22:43:01.278288 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.278306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.278454 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.278845 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.278848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.279621 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:01.279926 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.279937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.281198 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.281204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.282247 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.282260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.283214 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.283221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.283365 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.284152 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:01.284179 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:01.286729 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:01.290311 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:01.290327 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:01.290334 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:01.291000 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:01.291552 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:01.291560 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:01.291563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:01.291727 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:01.292328 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:01.292336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:01.292520 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.292709 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.293022 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":418,"threads_started":1}
I20260807 22:43:01.299705 22434 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:52932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:43:01.300995 22434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:01.309971 22735 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7566162cd44528261182e8d524f00 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cf9c0c81354475fb4fa666566070c8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:01.309772 22582 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7566162cd44528261182e8d524f00 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cf9c0c81354475fb4fa666566070c8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:01.310055 22885 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7566162cd44528261182e8d524f00 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cf9c0c81354475fb4fa666566070c8f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:01.310945 22582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7566162cd44528261182e8d524f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.310935 22735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7566162cd44528261182e8d524f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.310935 22885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7566162cd44528261182e8d524f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.318672 22983 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: Bootstrap starting.
I20260807 22:43:01.318939 22981 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: Bootstrap starting.
W20260807 22:43:01.319761 22811 tablet.cc:2369] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:01.320554 22983 tablet_bootstrap.cc:654] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.321132 22981 tablet_bootstrap.cc:654] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.324501 22982 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Bootstrap starting.
I20260807 22:43:01.327627 22982 tablet_bootstrap.cc:654] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.332901 22983 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: No bootstrap required, opened a new log
I20260807 22:43:01.333078 22983 ts_tablet_manager.cc:1397] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260807 22:43:01.333891 22983 raft_consensus.cc:348] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.334229 22983 raft_consensus.cc:374] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.334329 22983 raft_consensus.cc:729] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.334625 22982 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: No bootstrap required, opened a new log
I20260807 22:43:01.334679 22983 consensus_queue.cc:260] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.335477 22983 ts_tablet_manager.cc:1428] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:01.335400 22988 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.335985 22988 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.336618 22982 ts_tablet_manager.cc:1397] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260807 22:43:01.337777 22981 tablet_bootstrap.cc:492] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: No bootstrap required, opened a new log
I20260807 22:43:01.337978 22981 ts_tablet_manager.cc:1397] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.007s
I20260807 22:43:01.337795 22982 raft_consensus.cc:348] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.338320 22982 raft_consensus.cc:374] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.338469 22982 raft_consensus.cc:729] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.338819 22982 consensus_queue.cc:260] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.338919 22981 raft_consensus.cc:348] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.339203 22981 raft_consensus.cc:374] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.339326 22981 raft_consensus.cc:729] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.339388 22987 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.339594 22982 ts_tablet_manager.cc:1428] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 22:43:01.339737 22987 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.339645 22981 consensus_queue.cc:260] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.340358 22981 ts_tablet_manager.cc:1428] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:01.340247 22989 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.340729 22989 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.443542 22989 raft_consensus.cc:482] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:01.443852 22989 raft_consensus.cc:504] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.445727 22989 leader_election.cc:288] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:01.454952 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.446258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:38815 (local address 127.20.10.129:45019)
0807 22:43:01.446600 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.446610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.447664 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 22:43:01.447865 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.447874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.447943 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.448046 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.448761 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.449543 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.452169 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.453530 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.453759 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.453849 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.454466 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.454633 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 22:43:01.455021 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.446414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:58171 (local address 127.20.10.131:43213)
0807 22:43:01.446691 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.446700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.447558 (+ 858us) server_negotiation.cc:407] Connection header received
0807 22:43:01.447814 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.447818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.447887 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.448000 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.448840 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.449731 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.452318 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.453617 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.453769 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.453849 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.454505 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.454724 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:43:01.455078 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.446135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43213 (local address 127.20.10.130:58171)
0807 22:43:01.447421 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.447439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.447594 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.448019 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.448024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.448063 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.448627 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.448639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.449923 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.449928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.452116 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.452135 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.453974 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.453984 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.454286 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.454340 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.454444 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.454601 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":69,"thread_start_us":757,"threads_started":1}
I20260807 22:43:01.455855 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.446171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:45019 (local address 127.20.10.130:38815)
0807 22:43:01.447536 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.447556 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.447714 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.448061 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.448065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.448092 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.448558 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.448570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.449728 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.449740 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.451890 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.451909 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.453972 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.453980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.454183 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.454225 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.455204 (+ 979us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.455377 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"mutex_wait_us":51,"thread_start_us":839,"threads_started":1}
I20260807 22:43:01.455889 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7566162cd44528261182e8d524f00" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598" is_pre_election: true
I20260807 22:43:01.456238 22907 raft_consensus.cc:2454] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 0.
I20260807 22:43:01.456806 22685 leader_election.cc:302] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:01.457269 22989 raft_consensus.cc:2781] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:01.457415 22989 raft_consensus.cc:482] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:01.457487 22989 raft_consensus.cc:3037] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.457141 22611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7566162cd44528261182e8d524f00" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" is_pre_election: true
I20260807 22:43:01.457803 22611 raft_consensus.cc:2454] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 0.
I20260807 22:43:01.460184 22989 raft_consensus.cc:504] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.460819 22989 leader_election.cc:288] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 election: Requested vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:01.461508 22611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7566162cd44528261182e8d524f00" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
I20260807 22:43:01.461778 22611 raft_consensus.cc:3037] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.461750 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7566162cd44528261182e8d524f00" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598"
I20260807 22:43:01.462023 22907 raft_consensus.cc:3037] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.465335 22611 raft_consensus.cc:2454] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 1.
I20260807 22:43:01.465318 22907 raft_consensus.cc:2454] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 1.
I20260807 22:43:01.466090 22685 leader_election.cc:302] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:01.466544 22989 raft_consensus.cc:2781] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:01.467235 22989 raft_consensus.cc:686] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [term 1 LEADER]: Becoming Leader. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Running, Role: LEADER
I20260807 22:43:01.467907 22989 consensus_queue.cc:237] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.472182 23007 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: TxnStatusTablet state changed. Reason: New leader c7e76c67f2634dd6987cfb3119bac047. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.472711 23007 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:01.473572 23009 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:43:01.473814 23009 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:43:01.473842 22434 catalog_manager.cc:5495] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130). New cstate: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:01.501368 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.494233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.0.0.1:42892)
0807 22:43:01.494529 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.494553 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.494758 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.495285 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.495289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.495302 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:01.495522 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.495531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.496676 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.496682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.497967 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.497986 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.499231 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.499240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.499450 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.499530 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.500594 (+ 1064us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.500629 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.500807 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.501013 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 22:43:01.501394 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.494224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42892 (local address 127.20.10.130:41963)
0807 22:43:01.494717 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.494723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.494746 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:43:01.494999 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.495003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.495105 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.495260 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.495664 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.496534 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.498390 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.499008 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.499173 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.499770 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.500458 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.500637 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.500857 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.500964 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260807 22:43:01.511111 22907 raft_consensus.cc:1261] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Refusing update from remote peer c7e76c67f2634dd6987cfb3119bac047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.511102 22611 raft_consensus.cc:1261] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Refusing update from remote peer c7e76c67f2634dd6987cfb3119bac047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.512497 23007 consensus_queue.cc:1035] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.512892 22989 consensus_queue.cc:1035] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.524066 22987 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: TxnStatusTablet state changed. Reason: New leader c7e76c67f2634dd6987cfb3119bac047. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.524540 22987 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.530220 23012 mvcc.cc:205] Tried to move back new op lower bound from 7316040013856854016 to 7316040013697347584. Current Snapshot: MvccSnapshot[applied={T|T < 7316040013856854016}]
I20260807 22:43:01.530401 22988 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: TxnStatusTablet state changed. Reason: New leader c7e76c67f2634dd6987cfb3119bac047. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.530821 22988 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.534173 23013 mvcc.cc:205] Tried to move back new op lower bound from 7316040013856854016 to 7316040013697347584. Current Snapshot: MvccSnapshot[applied={T|T < 7316040013856854016}]
I20260807 22:43:01.536720 22989 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.537058 22989 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:01.537418 22989 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.537695 22989 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P c7e76c67f2634dd6987cfb3119bac047: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:01.540903 22988 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.541256 22988 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3a29098db2a94afca833c05c5c997598: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.541193 22987 tablet_replica.cc:439] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } } }
I20260807 22:43:01.541512 22987 tablet_replica.cc:442] T b8e7566162cd44528261182e8d524f00 P 3ba98cb2c87d421a9c63c9b8fb43d75f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:01.555927 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.548220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.20.10.129:45019)
0807 22:43:01.548445 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.548458 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.548603 (+ 145us) server_negotiation.cc:407] Connection header received
0807 22:43:01.548905 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.548910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.549035 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.549193 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.549682 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.550483 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.552817 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.553457 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.553596 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.554296 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.555010 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.555137 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.555415 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.555561 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:43:01.576258 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.569038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:56687 (local address 127.20.10.130:41963)
0807 22:43:01.569391 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.569399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.569839 (+ 440us) server_negotiation.cc:407] Connection header received
0807 22:43:01.570100 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.570105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.570218 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.570319 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.570921 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.572148 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.573926 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.574677 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.574815 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.575455 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.575621 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.575717 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.575851 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.575948 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:43:01.576476 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.568943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.20.10.129:56687)
0807 22:43:01.569690 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.569743 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.569949 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.570340 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.570345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.570358 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:01.570744 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.570757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.572459 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.572464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.573739 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.573754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.574917 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.574923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.575076 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.575146 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.575729 (+ 583us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.575739 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.575811 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.575945 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":390,"threads_started":1}
I20260807 22:43:01.589216 22906 raft_consensus.cc:1261] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Refusing update from remote peer 3ba98cb2c87d421a9c63c9b8fb43d75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.589408 22761 raft_consensus.cc:1261] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Refusing update from remote peer 3ba98cb2c87d421a9c63c9b8fb43d75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.589932 22992 consensus_queue.cc:1035] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.590781 22987 consensus_queue.cc:1035] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.598510 23014 mvcc.cc:205] Tried to move back new op lower bound from 7316040014181769216 to 7316040012683104256. Current Snapshot: MvccSnapshot[applied={T|T < 7316040014181769216}]
I20260807 22:43:01.629489 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.621265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:33715 (local address 127.20.10.129:45019)
0807 22:43:01.621471 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.621478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.622394 (+ 916us) server_negotiation.cc:407] Connection header received
0807 22:43:01.622746 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.622752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.622848 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.622921 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.623863 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.625100 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.627139 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.627859 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.627994 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.628607 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.628768 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.628858 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.629065 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.629164 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:43:01.629714 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.621453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:45019 (local address 127.20.10.130:33715)
0807 22:43:01.622269 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.622290 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.622474 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.623161 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.623166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.623179 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:01.623667 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.623682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.625329 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.625333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.626959 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.626975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.628090 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.628109 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.628302 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.628391 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.628927 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.628940 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.629028 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.629213 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":2667,"threads_started":1}
I20260807 22:43:01.679513 22433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:01.679863 22433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:01.684814 22433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Sending DeleteTablet for 3 replicas of tablet 509541329d404604ad6bb276e16e4176
I20260807 22:43:01.686605 22885 tablet_service.cc:1514] Processing DeleteTablet for tablet 509541329d404604ad6bb276e16e4176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57338
I20260807 22:43:01.687413 22735 tablet_service.cc:1514] Processing DeleteTablet for tablet 509541329d404604ad6bb276e16e4176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:42886
I20260807 22:43:01.688988 22582 tablet_service.cc:1514] Processing DeleteTablet for tablet 509541329d404604ad6bb276e16e4176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:59450
I20260807 22:43:01.696956 22433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:43:01.698972 23034 tablet_replica.cc:330] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: stopping tablet replica
I20260807 22:43:01.699795 23034 raft_consensus.cc:2229] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:01.700291 23034 raft_consensus.cc:2258] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:43:01.701203 22433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:01.702822 23034 ts_tablet_manager.cc:1905] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:01.713186 23034 ts_tablet_manager.cc:1918] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:01.713706 23034 log.cc:1196] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/509541329d404604ad6bb276e16e4176
I20260807 22:43:01.714385 23034 ts_tablet_manager.cc:1939] T 509541329d404604ad6bb276e16e4176 P 3a29098db2a94afca833c05c5c997598: Deleting consensus metadata
I20260807 22:43:01.716574 22885 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 (DEFAULT_TABLE table=test-table [id=d91577cfc51548438c99cc0bcc86265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.717216 22885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec014fff3df4c76afa4dde3d126ce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.717222 22734 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 (DEFAULT_TABLE table=test-table [id=d91577cfc51548438c99cc0bcc86265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.717808 22734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec014fff3df4c76afa4dde3d126ce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.723194 22581 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 (DEFAULT_TABLE table=test-table [id=d91577cfc51548438c99cc0bcc86265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.723892 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec014fff3df4c76afa4dde3d126ce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.733470 23035 tablet_replica.cc:330] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: stopping tablet replica
I20260807 22:43:01.734478 23036 tablet_replica.cc:330] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: stopping tablet replica
I20260807 22:43:01.735253 23035 raft_consensus.cc:2229] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:01.736420 22422 catalog_manager.cc:4841] TS 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213): tablet 509541329d404604ad6bb276e16e4176 (table test-table [id=ec1db060bd584fb3835ff4be46c15b11]) successfully deleted
I20260807 22:43:01.736977 22983 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Bootstrap starting.
I20260807 22:43:01.737946 23035 raft_consensus.cc:2258] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:01.735827 23036 raft_consensus.cc:2229] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:01.739068 23036 raft_consensus.cc:2258] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:01.739204 22983 tablet_bootstrap.cc:654] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.742970 22981 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Bootstrap starting.
I20260807 22:43:01.744843 22981 tablet_bootstrap.cc:654] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.745301 22982 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Bootstrap starting.
I20260807 22:43:01.747191 22982 tablet_bootstrap.cc:654] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.751869 23036 ts_tablet_manager.cc:1905] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:01.744848 23035 ts_tablet_manager.cc:1905] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:01.752451 22981 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: No bootstrap required, opened a new log
I20260807 22:43:01.752712 22981 ts_tablet_manager.cc:1397] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 22:43:01.753721 22981 raft_consensus.cc:348] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.754030 22981 raft_consensus.cc:374] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.754109 22981 raft_consensus.cc:729] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.754968 22983 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: No bootstrap required, opened a new log
I20260807 22:43:01.755038 22981 consensus_queue.cc:260] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.755203 22983 ts_tablet_manager.cc:1397] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.005s
I20260807 22:43:01.755710 22981 ts_tablet_manager.cc:1428] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 22:43:01.756381 22983 raft_consensus.cc:348] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.756635 22983 raft_consensus.cc:374] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.756717 22983 raft_consensus.cc:729] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.757140 22983 consensus_queue.cc:260] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.757853 22982 tablet_bootstrap.cc:492] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: No bootstrap required, opened a new log
I20260807 22:43:01.758044 22982 ts_tablet_manager.cc:1397] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260807 22:43:01.758937 22982 raft_consensus.cc:348] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.759241 22982 raft_consensus.cc:374] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.759361 22982 raft_consensus.cc:729] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.759634 22982 consensus_queue.cc:260] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.760442 22983 ts_tablet_manager.cc:1428] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 22:43:01.760967 22982 ts_tablet_manager.cc:1428] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:01.775898 23036 ts_tablet_manager.cc:1918] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:01.782706 23035 ts_tablet_manager.cc:1918] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:01.783073 23035 log.cc:1196] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/509541329d404604ad6bb276e16e4176
I20260807 22:43:01.783782 23035 ts_tablet_manager.cc:1939] T 509541329d404604ad6bb276e16e4176 P c7e76c67f2634dd6987cfb3119bac047: Deleting consensus metadata
I20260807 22:43:01.784042 23036 log.cc:1196] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/509541329d404604ad6bb276e16e4176
I20260807 22:43:01.784823 23036 ts_tablet_manager.cc:1939] T 509541329d404604ad6bb276e16e4176 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting consensus metadata
I20260807 22:43:01.787199 22418 catalog_manager.cc:4841] TS c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963): tablet 509541329d404604ad6bb276e16e4176 (table test-table [id=ec1db060bd584fb3835ff4be46c15b11]) successfully deleted
I20260807 22:43:01.789575 22418 catalog_manager.cc:4841] TS 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019): tablet 509541329d404604ad6bb276e16e4176 (table test-table [id=ec1db060bd584fb3835ff4be46c15b11]) successfully deleted
I20260807 22:43:01.846793 22988 raft_consensus.cc:482] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:01.847049 22988 raft_consensus.cc:504] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.848927 22988 leader_election.cc:288] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:01.859341 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.849406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:38669 (local address 127.20.10.129:45019)
0807 22:43:01.849645 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.849652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.850230 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:43:01.850611 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.850617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.850727 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.850796 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.851574 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.852737 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.856289 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.857863 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.858048 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.858131 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.858802 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.858968 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:43:01.860528 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.849266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:45019 (local address 127.20.10.131:38669)
0807 22:43:01.850105 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.850131 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.850426 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.851004 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.851009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.851038 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.851389 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.851402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.852964 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.852970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.856083 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.856101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.858345 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.858352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.858544 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.858583 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.859941 (+ 1358us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.860127 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":569,"threads_started":1}
I20260807 22:43:01.864115 22611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec014fff3df4c76afa4dde3d126ce85" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" is_pre_election: true
I20260807 22:43:01.864498 22611 raft_consensus.cc:2454] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a29098db2a94afca833c05c5c997598 in term 0.
I20260807 22:43:01.865777 22832 leader_election.cc:302] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, 3ba98cb2c87d421a9c63c9b8fb43d75f; no voters:
I20260807 22:43:01.866312 22988 raft_consensus.cc:2781] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:01.866465 22988 raft_consensus.cc:482] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:01.866577 22988 raft_consensus.cc:3037] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.869935 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.850838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:59091 (local address 127.20.10.130:41963)
0807 22:43:01.851047 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.851054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.861131 (+ 10077us) server_negotiation.cc:407] Connection header received
0807 22:43:01.861555 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.861562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.861646 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.861720 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.862325 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.863590 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.867052 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.868497 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.868661 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.868740 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.869388 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.869558 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:43:01.870369 22988 raft_consensus.cc:504] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.871006 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.850383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.20.10.131:59091)
0807 22:43:01.861014 (+ 10631us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.861037 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.861259 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.861923 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.861928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.861958 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.862137 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.862148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.863788 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.863794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.866831 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.866850 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.868942 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.868951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.869165 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.869201 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.870353 (+ 1152us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.870553 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10508,"thread_start_us":12754,"threads_started":1}
I20260807 22:43:01.871327 22988 leader_election.cc:288] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 election: Requested vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:01.872259 22611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec014fff3df4c76afa4dde3d126ce85" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
I20260807 22:43:01.872265 22761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec014fff3df4c76afa4dde3d126ce85" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047" is_pre_election: true
I20260807 22:43:01.872570 22611 raft_consensus.cc:3037] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.872644 22761 raft_consensus.cc:2454] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a29098db2a94afca833c05c5c997598 in term 0.
I20260807 22:43:01.872642 22759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec014fff3df4c76afa4dde3d126ce85" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047"
I20260807 22:43:01.872973 22759 raft_consensus.cc:3037] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.876765 22611 raft_consensus.cc:2454] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a29098db2a94afca833c05c5c997598 in term 1.
I20260807 22:43:01.876915 22759 raft_consensus.cc:2454] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a29098db2a94afca833c05c5c997598 in term 1.
I20260807 22:43:01.877741 22832 leader_election.cc:302] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, 3ba98cb2c87d421a9c63c9b8fb43d75f; no voters:
I20260807 22:43:01.878208 22988 raft_consensus.cc:2781] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:01.879086 22988 raft_consensus.cc:686] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 1 LEADER]: Becoming Leader. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Running, Role: LEADER
I20260807 22:43:01.879765 22988 consensus_queue.cc:237] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:01.884445 22433 catalog_manager.cc:5495] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a29098db2a94afca833c05c5c997598 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "3a29098db2a94afca833c05c5c997598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } health_report { overall_health: HEALTHY } } }
I20260807 22:43:01.952770 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.943557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.20.10.131:43213)
0807 22:43:01.943757 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.943764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.943792 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:43:01.944101 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.944108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.944250 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.944443 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.944963 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.946247 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.948553 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.949568 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.949770 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.950545 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.951795 (+ 1250us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.952025 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.952102 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.952255 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:43:01.996058 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.976989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:60409 (local address 127.20.10.130:41963)
0807 22:43:01.977307 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.977329 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.980955 (+ 3626us) server_negotiation.cc:407] Connection header received
0807 22:43:01.981261 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.981272 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.981404 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.981546 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.983407 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.985752 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.990541 (+ 4789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.991941 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.992095 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.993626 (+ 1531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.993882 (+ 256us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.994056 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.994399 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.994516 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 22:43:02.003364 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.975979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.20.10.131:60409)
0807 22:43:01.980801 (+ 4822us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.980823 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.981051 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.981803 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.981813 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.981832 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:01.983025 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.983059 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.986170 (+ 3111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.986177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.990197 (+ 4020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.990237 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.992633 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.992646 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.993079 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.993214 (+ 135us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.994160 (+ 946us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.994176 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.995023 (+ 847us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.995410 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4524,"thread_start_us":3976,"threads_started":1}
I20260807 22:43:02.050645 22759 raft_consensus.cc:1261] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Refusing update from remote peer 3a29098db2a94afca833c05c5c997598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:02.052093 22611 raft_consensus.cc:1261] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Refusing update from remote peer 3a29098db2a94afca833c05c5c997598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:02.052142 22988 consensus_queue.cc:1035] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:02.052810 22988 consensus_queue.cc:1035] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:02.171444 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.149938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49333 (local address 127.20.10.131:43213)
0807 22:43:02.150195 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.150202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.159339 (+ 9137us) server_negotiation.cc:407] Connection header received
0807 22:43:02.159800 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.159807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.159915 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.160068 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:02.160738 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.162579 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.166833 (+ 4254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.168538 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.168744 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.170109 (+ 1365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:02.170326 (+ 217us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:02.170421 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.170746 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.170864 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 22:43:02.171499 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.150415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43213 (local address 127.20.10.130:49333)
0807 22:43:02.159145 (+ 8730us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.159186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.159526 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.160108 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.160114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.160129 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:02.160524 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.160539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.163011 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.163031 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.166365 (+ 3334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.166401 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.169132 (+ 2731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.169148 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.169549 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.169653 (+ 104us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:02.170472 (+ 819us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:02.170487 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.170592 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.170839 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8496,"thread_start_us":8311,"threads_started":1}
W20260807 22:43:02.174039 23013 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:43:03.845276 22433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:03.845546 22433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:03.851631 22433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Sending DeleteTablet for 3 replicas of tablet 3ec014fff3df4c76afa4dde3d126ce85
I20260807 22:43:03.852847 22885 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:03 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57338
I20260807 22:43:03.852860 22581 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:03 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:59450
I20260807 22:43:03.853063 22734 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec014fff3df4c76afa4dde3d126ce85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:03 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:42886
I20260807 22:43:03.864414 23075 tablet_replica.cc:330] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: stopping tablet replica
I20260807 22:43:03.864827 23075 raft_consensus.cc:2229] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:03.865715 23077 tablet_replica.cc:330] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: stopping tablet replica
I20260807 22:43:03.866091 23075 raft_consensus.cc:2258] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:03.866173 23077 raft_consensus.cc:2229] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:03.866606 23077 raft_consensus.cc:2258] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:03.867756 23075 ts_tablet_manager.cc:1905] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:03.868119 23077 ts_tablet_manager.cc:1905] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:03.873703 23076 tablet_replica.cc:330] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: stopping tablet replica
I20260807 22:43:03.874156 23076 raft_consensus.cc:2229] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:03.874696 23076 raft_consensus.cc:2258] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:03.876209 23076 ts_tablet_manager.cc:1905] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:43:03 dist-test-slave-csg5 krb5kdc[22386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142583, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260807 22:43:03.962333 23077 ts_tablet_manager.cc:1918] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:03.962357 23076 ts_tablet_manager.cc:1918] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:03.962558 23077 log.cc:1196] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/3ec014fff3df4c76afa4dde3d126ce85
I20260807 22:43:03.962612 23076 log.cc:1196] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/3ec014fff3df4c76afa4dde3d126ce85
I20260807 22:43:03.962844 23075 ts_tablet_manager.cc:1918] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:03.963130 23075 log.cc:1196] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/3ec014fff3df4c76afa4dde3d126ce85
I20260807 22:43:03.963256 23077 ts_tablet_manager.cc:1939] T 3ec014fff3df4c76afa4dde3d126ce85 P c7e76c67f2634dd6987cfb3119bac047: Deleting consensus metadata
I20260807 22:43:03.963380 23076 ts_tablet_manager.cc:1939] T 3ec014fff3df4c76afa4dde3d126ce85 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting consensus metadata
W20260807 22:43:03.963507 20522 mini_kdc.cc:317] Time spent kinit for test-user: real 0.110s user 0.003s sys 0.015s
I20260807 22:43:03.963948 23075 ts_tablet_manager.cc:1939] T 3ec014fff3df4c76afa4dde3d126ce85 P 3a29098db2a94afca833c05c5c997598: Deleting consensus metadata
Aug 07 22:43:03 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142583, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:03.989388 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:03.967630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.20.10.190:35507)
0807 22:43:03.968868 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:03.968873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:03.968907 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:43:03.968967 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:03.968970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:03.969037 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:03.969181 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:03.971250 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.972453 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.974336 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.975192 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.975361 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.981052 (+ 5691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:03.981059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:03.981063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:03.981112 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:03.985154 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:03.986188 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:03.986195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:03.986197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:03.986543 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:03.987283 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:03.987287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:03.987289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:03.987831 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:03.988082 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:03.988567 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 22:43:03.988812 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":802,"threads_started":1}
I20260807 22:43:03.994558 22433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:03.995460 22433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:04.004899 22581 tablet_service.cc:1467] Processing CreateTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 (DEFAULT_TABLE table=test-table [id=5561eece823e4261b455efb924299b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.004915 22734 tablet_service.cc:1467] Processing CreateTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 (DEFAULT_TABLE table=test-table [id=5561eece823e4261b455efb924299b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.005064 22885 tablet_service.cc:1467] Processing CreateTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 (DEFAULT_TABLE table=test-table [id=5561eece823e4261b455efb924299b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.005548 22734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 774c18e87482430ca1eab8c1ca0ac7b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.005564 22885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 774c18e87482430ca1eab8c1ca0ac7b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.005574 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 774c18e87482430ca1eab8c1ca0ac7b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.018606 23086 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Bootstrap starting.
I20260807 22:43:04.019105 23087 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Bootstrap starting.
I20260807 22:43:04.021174 23087 tablet_bootstrap.cc:654] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.021178 23088 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Bootstrap starting.
I20260807 22:43:04.023078 23086 tablet_bootstrap.cc:654] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.024072 23088 tablet_bootstrap.cc:654] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.027098 23086 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: No bootstrap required, opened a new log
I20260807 22:43:04.027096 23087 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: No bootstrap required, opened a new log
I20260807 22:43:04.027316 23087 ts_tablet_manager.cc:1397] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260807 22:43:04.027313 23086 ts_tablet_manager.cc:1397] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 22:43:04.027704 23088 tablet_bootstrap.cc:492] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: No bootstrap required, opened a new log
I20260807 22:43:04.027905 23088 ts_tablet_manager.cc:1397] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 22:43:04.028568 23086 raft_consensus.cc:348] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.028548 23087 raft_consensus.cc:348] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.028904 23086 raft_consensus.cc:374] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.028987 23087 raft_consensus.cc:374] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.029035 23086 raft_consensus.cc:729] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.029305 23087 raft_consensus.cc:729] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.029173 23088 raft_consensus.cc:348] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.029479 23088 raft_consensus.cc:374] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.029621 23088 raft_consensus.cc:729] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.029716 23086 consensus_queue.cc:260] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.029801 23087 consensus_queue.cc:260] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.029943 23088 consensus_queue.cc:260] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.030736 23086 ts_tablet_manager.cc:1428] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:04.031688 23088 ts_tablet_manager.cc:1428] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 22:43:04.032596 22422 catalog_manager.cc:4841] TS 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213): tablet 3ec014fff3df4c76afa4dde3d126ce85 (table test-table [id=d91577cfc51548438c99cc0bcc86265b]) successfully deleted
I20260807 22:43:04.032974 23087 ts_tablet_manager.cc:1428] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260807 22:43:04.034196 22418 catalog_manager.cc:4841] TS c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963): tablet 3ec014fff3df4c76afa4dde3d126ce85 (table test-table [id=d91577cfc51548438c99cc0bcc86265b]) successfully deleted
I20260807 22:43:04.035086 22418 catalog_manager.cc:4841] TS 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019): tablet 3ec014fff3df4c76afa4dde3d126ce85 (table test-table [id=d91577cfc51548438c99cc0bcc86265b]) successfully deleted
I20260807 22:43:04.080092 23092 raft_consensus.cc:482] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:04.080598 23092 raft_consensus.cc:504] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.081650 23092 leader_election.cc:288] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:04.082091 23065 raft_consensus.cc:482] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:04.082396 23065 raft_consensus.cc:504] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.082613 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598" is_pre_election: true
I20260807 22:43:04.082708 22759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047" is_pre_election: true
I20260807 22:43:04.083093 22906 raft_consensus.cc:2454] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 0.
I20260807 22:43:04.083165 22759 raft_consensus.cc:2454] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 0.
I20260807 22:43:04.083628 23065 leader_election.cc:288] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:04.083820 22530 leader_election.cc:302] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, 3ba98cb2c87d421a9c63c9b8fb43d75f, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:04.084456 22611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" is_pre_election: true
I20260807 22:43:04.084448 22759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047" is_pre_election: true
I20260807 22:43:04.084830 22759 raft_consensus.cc:2454] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a29098db2a94afca833c05c5c997598 in term 0.
I20260807 22:43:04.084842 23092 raft_consensus.cc:2781] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:04.085426 23092 raft_consensus.cc:482] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:04.085428 22832 leader_election.cc:302] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:04.085613 23092 raft_consensus.cc:3037] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.085881 23065 raft_consensus.cc:2781] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:04.086090 23065 raft_consensus.cc:482] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:04.086252 23065 raft_consensus.cc:3037] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.155045 23092 raft_consensus.cc:504] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.154999 23065 raft_consensus.cc:504] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.156010 22611 raft_consensus.cc:2379] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a29098db2a94afca833c05c5c997598 in current term 1: Already voted for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in this term.
I20260807 22:43:04.156947 22610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
I20260807 22:43:04.157325 23065 leader_election.cc:288] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 election: Requested vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963)
I20260807 22:43:04.157379 22610 raft_consensus.cc:2379] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a29098db2a94afca833c05c5c997598 in current term 1: Already voted for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in this term.
I20260807 22:43:04.157454 23092 leader_election.cc:288] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 election: Requested vote from peers c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:04.157881 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598"
I20260807 22:43:04.158093 22759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047"
I20260807 22:43:04.158253 22906 raft_consensus.cc:2379] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in current term 1: Already voted for candidate 3a29098db2a94afca833c05c5c997598 in this term.
I20260807 22:43:04.158506 22759 raft_consensus.cc:3037] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.159261 22761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "774c18e87482430ca1eab8c1ca0ac7b8" candidate_uuid: "3a29098db2a94afca833c05c5c997598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e76c67f2634dd6987cfb3119bac047"
I20260807 22:43:04.160238 22832 leader_election.cc:302] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a29098db2a94afca833c05c5c997598; no voters: 3ba98cb2c87d421a9c63c9b8fb43d75f, c7e76c67f2634dd6987cfb3119bac047
I20260807 22:43:04.160717 23065 raft_consensus.cc:2726] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 22:43:04.257539 22759 raft_consensus.cc:2454] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ba98cb2c87d421a9c63c9b8fb43d75f in term 1.
I20260807 22:43:04.258460 22530 leader_election.cc:302] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ba98cb2c87d421a9c63c9b8fb43d75f, c7e76c67f2634dd6987cfb3119bac047; no voters: 3a29098db2a94afca833c05c5c997598
I20260807 22:43:04.259043 23092 raft_consensus.cc:2781] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:04.259907 23092 raft_consensus.cc:686] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 LEADER]: Becoming Leader. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Running, Role: LEADER
I20260807 22:43:04.260399 23092 consensus_queue.cc:237] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.264755 22433 catalog_manager.cc:5495] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129). New cstate: current_term: 1 leader_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:04.354566 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:04.344688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.20.10.129:45019)
0807 22:43:04.345661 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:04.345666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:04.345691 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:04.345753 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:04.345757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:04.345824 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:04.345911 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:04.346502 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.347663 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.351038 (+ 3375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.352132 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.352328 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:04.353279 (+ 951us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:04.353427 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:04.353507 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:04.353908 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 22:43:04.354012 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":666,"threads_started":1}
I20260807 22:43:04.368388 22759 raft_consensus.cc:1261] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Refusing update from remote peer 3ba98cb2c87d421a9c63c9b8fb43d75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:04.368523 22906 raft_consensus.cc:1261] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Refusing update from remote peer 3ba98cb2c87d421a9c63c9b8fb43d75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:04.369249 23093 consensus_queue.cc:1035] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:04.369839 23092 consensus_queue.cc:1035] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:04.439316 22434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:04.439767 22434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:04.443985 22434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Sending DeleteTablet for 3 replicas of tablet 774c18e87482430ca1eab8c1ca0ac7b8
I20260807 22:43:04.445365 22581 tablet_service.cc:1514] Processing DeleteTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:04 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:59450
I20260807 22:43:04.445420 22885 tablet_service.cc:1514] Processing DeleteTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:04 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57338
I20260807 22:43:04.445781 23076 tablet_replica.cc:330] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: stopping tablet replica
I20260807 22:43:04.445859 23075 tablet_replica.cc:330] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: stopping tablet replica
I20260807 22:43:04.445712 22734 tablet_service.cc:1514] Processing DeleteTablet for tablet 774c18e87482430ca1eab8c1ca0ac7b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:04 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:42886
I20260807 22:43:04.446132 23076 raft_consensus.cc:2229] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:04.446200 23075 raft_consensus.cc:2229] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:04.446512 23077 tablet_replica.cc:330] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: stopping tablet replica
I20260807 22:43:04.446496 23075 raft_consensus.cc:2258] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:04.446700 23076 raft_consensus.cc:2258] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:04.446835 23077 raft_consensus.cc:2229] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:04.447183 23077 raft_consensus.cc:2258] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:04.446998 22434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:04.448077 22434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:04.448150 23075 ts_tablet_manager.cc:1905] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:04.448654 23077 ts_tablet_manager.cc:1905] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:04.448859 23076 ts_tablet_manager.cc:1905] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:04.457379 23077 ts_tablet_manager.cc:1918] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:04.457623 23077 log.cc:1196] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/774c18e87482430ca1eab8c1ca0ac7b8
I20260807 22:43:04.458185 23077 ts_tablet_manager.cc:1939] T 774c18e87482430ca1eab8c1ca0ac7b8 P c7e76c67f2634dd6987cfb3119bac047: Deleting consensus metadata
I20260807 22:43:04.458554 22885 tablet_service.cc:1467] Processing CreateTablet for tablet a4e46c6a59644dbca10d779b130c2236 (DEFAULT_TABLE table=test-table [id=1e56316997dc4d1280525d8aa5715966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.458557 22734 tablet_service.cc:1467] Processing CreateTablet for tablet a4e46c6a59644dbca10d779b130c2236 (DEFAULT_TABLE table=test-table [id=1e56316997dc4d1280525d8aa5715966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.459060 22734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e46c6a59644dbca10d779b130c2236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.459096 22885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e46c6a59644dbca10d779b130c2236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.459518 22581 tablet_service.cc:1467] Processing CreateTablet for tablet a4e46c6a59644dbca10d779b130c2236 (DEFAULT_TABLE table=test-table [id=1e56316997dc4d1280525d8aa5715966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:04.459892 23075 ts_tablet_manager.cc:1918] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:04.460017 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e46c6a59644dbca10d779b130c2236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:04.460222 23075 log.cc:1196] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/774c18e87482430ca1eab8c1ca0ac7b8
I20260807 22:43:04.460834 23075 ts_tablet_manager.cc:1939] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3a29098db2a94afca833c05c5c997598: Deleting consensus metadata
I20260807 22:43:04.462491 22422 catalog_manager.cc:4841] TS 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213): tablet 774c18e87482430ca1eab8c1ca0ac7b8 (table test-table [id=5561eece823e4261b455efb924299b8a]) successfully deleted
I20260807 22:43:04.465332 23076 ts_tablet_manager.cc:1918] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:04.465780 23076 log.cc:1196] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/774c18e87482430ca1eab8c1ca0ac7b8
I20260807 22:43:04.466569 23076 ts_tablet_manager.cc:1939] T 774c18e87482430ca1eab8c1ca0ac7b8 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting consensus metadata
I20260807 22:43:04.468429 22418 catalog_manager.cc:4841] TS 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019): tablet 774c18e87482430ca1eab8c1ca0ac7b8 (table test-table [id=5561eece823e4261b455efb924299b8a]) successfully deleted
I20260807 22:43:04.468647 23087 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Bootstrap starting.
I20260807 22:43:04.468966 23086 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Bootstrap starting.
I20260807 22:43:04.469453 23088 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Bootstrap starting.
I20260807 22:43:04.470800 22418 catalog_manager.cc:4841] TS c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963): tablet 774c18e87482430ca1eab8c1ca0ac7b8 (table test-table [id=5561eece823e4261b455efb924299b8a]) successfully deleted
I20260807 22:43:04.471073 23086 tablet_bootstrap.cc:654] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.471551 23087 tablet_bootstrap.cc:654] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.471653 23088 tablet_bootstrap.cc:654] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:04.474849 23086 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: No bootstrap required, opened a new log
I20260807 22:43:04.475106 23086 ts_tablet_manager.cc:1397] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260807 22:43:04.476279 23087 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: No bootstrap required, opened a new log
I20260807 22:43:04.476656 23087 ts_tablet_manager.cc:1397] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260807 22:43:04.476461 23086 raft_consensus.cc:348] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.476989 23086 raft_consensus.cc:374] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.477025 23088 tablet_bootstrap.cc:492] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: No bootstrap required, opened a new log
I20260807 22:43:04.477358 23086 raft_consensus.cc:729] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a29098db2a94afca833c05c5c997598, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.477414 23088 ts_tablet_manager.cc:1397] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:43:04.478904 23087 raft_consensus.cc:348] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.479063 23086 consensus_queue.cc:260] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.479071 23088 raft_consensus.cc:348] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.479633 23088 raft_consensus.cc:374] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.479699 23087 raft_consensus.cc:374] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:04.479799 23088 raft_consensus.cc:729] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.480063 23087 raft_consensus.cc:729] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba98cb2c87d421a9c63c9b8fb43d75f, State: Initialized, Role: FOLLOWER
I20260807 22:43:04.480324 23088 consensus_queue.cc:260] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.480576 23087 consensus_queue.cc:260] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.482097 23088 ts_tablet_manager.cc:1428] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:43:04.483459 23086 ts_tablet_manager.cc:1428] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Time spent starting tablet: real 0.008s user 0.003s sys 0.001s
I20260807 22:43:04.484405 23087 ts_tablet_manager.cc:1428] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260807 22:43:04.632916 23096 raft_consensus.cc:482] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:04.633282 23096 raft_consensus.cc:504] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.634588 23096 leader_election.cc:288] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:04.635514 22610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e46c6a59644dbca10d779b130c2236" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" is_pre_election: true
I20260807 22:43:04.635535 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e46c6a59644dbca10d779b130c2236" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598" is_pre_election: true
I20260807 22:43:04.635906 22906 raft_consensus.cc:2454] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 0.
I20260807 22:43:04.635959 22610 raft_consensus.cc:2454] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 0.
I20260807 22:43:04.636515 22685 leader_election.cc:302] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a29098db2a94afca833c05c5c997598, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:04.636963 23096 raft_consensus.cc:2781] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:04.637226 23096 raft_consensus.cc:482] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:04.637346 23096 raft_consensus.cc:3037] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.640522 23096 raft_consensus.cc:504] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.641593 23096 leader_election.cc:288] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 election: Requested vote from peers 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019), 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213)
I20260807 22:43:04.642210 22610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e46c6a59644dbca10d779b130c2236" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f"
I20260807 22:43:04.642334 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e46c6a59644dbca10d779b130c2236" candidate_uuid: "c7e76c67f2634dd6987cfb3119bac047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a29098db2a94afca833c05c5c997598"
I20260807 22:43:04.642486 22610 raft_consensus.cc:3037] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.642644 22906 raft_consensus.cc:3037] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:04.645305 22610 raft_consensus.cc:2454] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 1.
I20260807 22:43:04.645834 22906 raft_consensus.cc:2454] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7e76c67f2634dd6987cfb3119bac047 in term 1.
I20260807 22:43:04.645959 22682 leader_election.cc:302] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ba98cb2c87d421a9c63c9b8fb43d75f, c7e76c67f2634dd6987cfb3119bac047; no voters:
I20260807 22:43:04.646502 23096 raft_consensus.cc:2781] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:04.646915 23096 raft_consensus.cc:686] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 1 LEADER]: Becoming Leader. State: Replica: c7e76c67f2634dd6987cfb3119bac047, State: Running, Role: LEADER
I20260807 22:43:04.647320 23096 consensus_queue.cc:237] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } }
I20260807 22:43:04.651713 22434 catalog_manager.cc:5495] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130). New cstate: current_term: 1 leader_uuid: "c7e76c67f2634dd6987cfb3119bac047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7e76c67f2634dd6987cfb3119bac047" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:04.696842 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:04.690844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.20.10.130:41963)
0807 22:43:04.691494 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:04.691498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:04.691518 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:04.691592 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:04.691596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:04.691660 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:04.691778 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:04.692320 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.693139 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.694797 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.695252 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.695408 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:04.695993 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:04.696123 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:04.696231 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:04.696405 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 22:43:04.696496 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":357,"threads_started":1}
I20260807 22:43:04.717026 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:04.709748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49033 (local address 127.20.10.130:41963)
0807 22:43:04.709945 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:04.709953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:04.710364 (+ 411us) server_negotiation.cc:407] Connection header received
0807 22:43:04.710565 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:04.710570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:04.710639 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:04.710832 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:04.711635 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.712883 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.714840 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.715368 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.715501 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:04.716029 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:04.716151 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:04.716326 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:04.716524 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 22:43:04.716706 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:43:04.717788 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:04.709157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41963 (local address 127.20.10.130:49033)
0807 22:43:04.710240 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:04.710256 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:04.710413 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:04.710862 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:04.710865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:04.710876 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:04.711412 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:04.711424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.713034 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.713037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:04.714482 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:04.714499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:04.715627 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:04.715632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:04.715812 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:04.715856 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:04.716335 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:04.716348 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:43:04.716443 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:43:04.716580 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":726,"threads_started":1}
I20260807 22:43:04.728411 22610 raft_consensus.cc:1261] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Refusing update from remote peer c7e76c67f2634dd6987cfb3119bac047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:04.728613 22906 raft_consensus.cc:1261] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Refusing update from remote peer c7e76c67f2634dd6987cfb3119bac047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:04.729154 23056 consensus_queue.cc:1035] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba98cb2c87d421a9c63c9b8fb43d75f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:04.729699 23108 consensus_queue.cc:1035] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a29098db2a94afca833c05c5c997598" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:43:04.773168 23095 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:43:06.405419 22435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:06.405786 22435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:06.413759 22435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6369e199f7a342439ada581f94bd5b82: Sending DeleteTablet for 3 replicas of tablet a4e46c6a59644dbca10d779b130c2236
I20260807 22:43:06.415151 22581 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e46c6a59644dbca10d779b130c2236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:06 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:59450
I20260807 22:43:06.415268 22885 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e46c6a59644dbca10d779b130c2236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:06 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57338
I20260807 22:43:06.415354 22735 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e46c6a59644dbca10d779b130c2236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:06 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:42886
I20260807 22:43:06.416375 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22514
I20260807 22:43:06.419775 23149 tablet_replica.cc:330] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: stopping tablet replica
I20260807 22:43:06.420195 23149 raft_consensus.cc:2229] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:06.421228 23149 raft_consensus.cc:2258] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:06.422329 23148 tablet_replica.cc:330] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: stopping tablet replica
I20260807 22:43:06.422497 23147 tablet_replica.cc:330] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: stopping tablet replica
I20260807 22:43:06.422636 23148 raft_consensus.cc:2229] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:06.422863 23147 raft_consensus.cc:2229] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:06.423130 23148 raft_consensus.cc:2258] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:06.423249 23147 raft_consensus.cc:2258] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:06.424433 23149 ts_tablet_manager.cc:1905] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:06.424654 23147 ts_tablet_manager.cc:1905] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:06.433460 23148 ts_tablet_manager.cc:1905] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:06.435935 23147 ts_tablet_manager.cc:1918] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:06.436142 23147 log.cc:1196] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/a4e46c6a59644dbca10d779b130c2236
I20260807 22:43:06.436448 23149 ts_tablet_manager.cc:1918] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:06.436651 23149 log.cc:1196] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/a4e46c6a59644dbca10d779b130c2236
I20260807 22:43:06.437076 23147 ts_tablet_manager.cc:1939] T a4e46c6a59644dbca10d779b130c2236 P 3ba98cb2c87d421a9c63c9b8fb43d75f: Deleting consensus metadata
I20260807 22:43:06.437454 23149 ts_tablet_manager.cc:1939] T a4e46c6a59644dbca10d779b130c2236 P c7e76c67f2634dd6987cfb3119bac047: Deleting consensus metadata
I20260807 22:43:06.438621 22418 catalog_manager.cc:4841] TS 3ba98cb2c87d421a9c63c9b8fb43d75f (127.20.10.129:45019): tablet a4e46c6a59644dbca10d779b130c2236 (table test-table [id=1e56316997dc4d1280525d8aa5715966]) successfully deleted
I20260807 22:43:06.439056 22418 catalog_manager.cc:4841] TS c7e76c67f2634dd6987cfb3119bac047 (127.20.10.130:41963): tablet a4e46c6a59644dbca10d779b130c2236 (table test-table [id=1e56316997dc4d1280525d8aa5715966]) successfully deleted
I20260807 22:43:06.441650 23148 ts_tablet_manager.cc:1918] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:43:06.441895 23148 log.cc:1196] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/a4e46c6a59644dbca10d779b130c2236
I20260807 22:43:06.442554 23148 ts_tablet_manager.cc:1939] T a4e46c6a59644dbca10d779b130c2236 P 3a29098db2a94afca833c05c5c997598: Deleting consensus metadata
I20260807 22:43:06.443988 22422 catalog_manager.cc:4841] TS 3a29098db2a94afca833c05c5c997598 (127.20.10.131:43213): tablet a4e46c6a59644dbca10d779b130c2236 (table test-table [id=1e56316997dc4d1280525d8aa5715966]) successfully deleted
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142583, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:06.462090 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.419662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.20.10.129:45019)
0807 22:43:06.431565 (+ 11903us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.431573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.432787 (+ 1214us) server_negotiation.cc:407] Connection header received
0807 22:43:06.432895 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.432901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.433045 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.433198 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.436472 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.439092 (+ 2620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.440974 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.441961 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.442131 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.452554 (+ 10423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.452583 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.452600 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.452651 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.457789 (+ 5138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.458936 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.458944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.458947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.459201 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.459855 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.459860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.459862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.460435 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.460743 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.461240 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.461468 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11662,"thread_start_us":12969,"threads_started":1}
W20260807 22:43:06.463130 22656 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59476
W20260807 22:43:06.464080 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22514: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:06.464411 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 22514
I20260807 22:43:06.498085 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22666
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142583, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:43:06.517870 23153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.499050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.20.10.130:41963)
0807 22:43:06.499989 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.499994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.500014 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:06.500078 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.500082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.500149 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.500267 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.502136 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.503322 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.504843 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.505468 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.505583 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.510870 (+ 5287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.510897 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.510920 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.510981 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.514339 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.515411 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.515419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.515422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.515615 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.516035 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.516039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.516042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.516508 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.516814 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.517315 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.517537 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":624,"threads_started":1}
W20260807 22:43:06.518787 22805 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42918
W20260807 22:43:06.519502 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22666: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:06.519693 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 22666
I20260807 22:43:06.556633 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22815
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[22386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142583, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:43:06.577914 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.557568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.20.10.131:43213)
0807 22:43:06.558216 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.558221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.558238 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:06.558307 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.558311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.558384 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.558539 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.560311 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.561288 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.563175 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.563665 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.563848 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.570147 (+ 6299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.570168 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.570187 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.570238 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.574121 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.575288 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.575297 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.575300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.575480 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.575987 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.575991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.575994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.576455 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.576832 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.577189 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.577463 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":322,"threads_started":1}
W20260807 22:43:06.579008 22956 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57360
W20260807 22:43:06.579890 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22815: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:06.580088 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 22815
I20260807 22:43:06.611470 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 22401
I20260807 22:43:06.612926 22480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:06.745766 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 22401
2026-08-07T22:43:06Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9697 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[23159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[23159](info): set up 2 sockets
Aug 07 22:43:06 dist-test-slave-csg5 krb5kdc[23159](info): commencing operation
krb5kdc: starting...
W20260807 22:43:08.001468 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:08 dist-test-slave-csg5 krb5kdc[23159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:08.269234 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46059
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:08.643482 23174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:08.643808 23174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:08.643987 23174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:08.651594 23174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:08.651724 23174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:08.651818 23174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:08.651906 23174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:08.651971 23174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:08.662832 23174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:08.664412 23174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:08.666601 23174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:08.681473 23179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.681424 23182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.681550 23184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.683166 23174 server_base.cc:1034] running on GCE node
I20260807 22:43:08.684171 23174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:08.685911 23174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:08.687140 23174 hybrid_clock.cc:648] HybridClock initialized: now 1786142588687094 us; error 42 us; skew 500 ppm
I20260807 22:43:08.687561 23174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:08 dist-test-slave-csg5 krb5kdc[23159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:08.691601 23174 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:08.693605 23174 webserver.cc:460] Webserver started at http://127.20.10.190:40923/ using document root <none> and password file <none>
I20260807 22:43:08.694427 23174 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:08.694615 23174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:08.695026 23174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:08.698700 23174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "3d657042f46c4bad9f0dbadf5b153cd6"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-csg5"
server_key: "7b54e92ebabaad79791a956b792984d2"
server_key_iv: "3a08e8663e3e853e542e9462d8d470b2"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.699636 23174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "3d657042f46c4bad9f0dbadf5b153cd6"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-csg5"
server_key: "7b54e92ebabaad79791a956b792984d2"
server_key_iv: "3a08e8663e3e853e542e9462d8d470b2"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.706279 23174 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:43:08.711306 23192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.713083 23174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:43:08.713351 23174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "3d657042f46c4bad9f0dbadf5b153cd6"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-csg5"
server_key: "7b54e92ebabaad79791a956b792984d2"
server_key_iv: "3a08e8663e3e853e542e9462d8d470b2"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.713606 23174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.739811 23174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.745157 23174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.745520 23174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.767179 23174 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46059
I20260807 22:43:08.767225 23247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46059 every 8 connection(s)
I20260807 22:43:08.768659 23174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:08.772774 23248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:08.776845 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23174
I20260807 22:43:08.777104 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:08.777880 20522 external_mini_cluster.cc:1428] Setting key 517ec304909087535330bf415303aef8
I20260807 22:43:08.784775 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6: Bootstrap starting.
I20260807 22:43:08.788813 23248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:08.790200 23248 log.cc:824] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:08.794703 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6: No bootstrap required, opened a new log
Aug 07 22:43:08 dist-test-slave-csg5 krb5kdc[23159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:08.804800 23248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } }
I20260807 22:43:08.805338 23248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:08.805516 23248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d657042f46c4bad9f0dbadf5b153cd6, State: Initialized, Role: FOLLOWER
I20260807 22:43:08.806169 23248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } }
I20260807 22:43:08.806481 23248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:08.806726 23248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:08.806916 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.780223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.20.10.190:46059)
0807 22:43:08.781213 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.781226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.781294 (+ 68us) server_negotiation.cc:407] Connection header received
0807 22:43:08.783553 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.783581 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.784104 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.784483 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.786604 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.788309 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.790282 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.791601 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.791840 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.797398 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.797413 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.797434 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.797493 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.801589 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.802501 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.802517 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.802525 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.802823 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.803295 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.803310 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.803312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.803965 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.804319 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.804831 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.805389 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":449,"threads_started":1}
I20260807 22:43:08.806962 23248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:08.810370 23248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } }
I20260807 22:43:08.811002 23248 leader_election.cc:302] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d657042f46c4bad9f0dbadf5b153cd6; no voters:
I20260807 22:43:08.811694 23248 leader_election.cc:288] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:08.811857 23253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:08.812690 23253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d657042f46c4bad9f0dbadf5b153cd6, State: Running, Role: LEADER
I20260807 22:43:08.813416 23253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } }
I20260807 22:43:08.814194 23248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:08.816704 23254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } } }
I20260807 22:43:08.816881 23255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d657042f46c4bad9f0dbadf5b153cd6. Latest consensus state: current_term: 1 leader_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d657042f46c4bad9f0dbadf5b153cd6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46059 } } }
I20260807 22:43:08.817279 23255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:08.817286 23254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:08.819859 23259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:08.825913 23259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:08.835510 23259 catalog_manager.cc:1347] Generated new cluster ID: 932ca3b7c3b94017ae21e0fd9cc25491
I20260807 22:43:08.835662 23259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:08.893380 23259 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:08.895136 23259 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:08.915560 23259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d657042f46c4bad9f0dbadf5b153cd6: Generated new TSK 0
I20260807 22:43:08.916710 23259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:43:08.987906 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:09.368599 23276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.368907 23276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.369006 23276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.376621 23276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.376752 23276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.376874 23276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:09.387193 23276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:09.388691 23276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.390071 23276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.405274 23281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.405694 23282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.405364 23285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.406296 23276 server_base.cc:1034] running on GCE node
I20260807 22:43:09.407054 23276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.408155 23276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.409390 23276 hybrid_clock.cc:648] HybridClock initialized: now 1786142589409309 us; error 69 us; skew 500 ppm
I20260807 22:43:09.409781 23276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:09 dist-test-slave-csg5 krb5kdc[23159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.413605 23276 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:43:09.415964 23276 webserver.cc:460] Webserver started at http://127.20.10.129:36995/ using document root <none> and password file <none>
I20260807 22:43:09.416772 23276 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.416942 23276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.417310 23276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:09.421020 23276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "176504ac4ba340f08947600166b3ef93"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-csg5"
server_key: "3ea1718717f6564977edadb531a3ac8f"
server_key_iv: "68ef278f0b94a84cd2c81375d01fe58e"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.421969 23276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "176504ac4ba340f08947600166b3ef93"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-csg5"
server_key: "3ea1718717f6564977edadb531a3ac8f"
server_key_iv: "68ef278f0b94a84cd2c81375d01fe58e"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.428349 23276 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:43:09.433754 23292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.435549 23276 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:09.435825 23276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "176504ac4ba340f08947600166b3ef93"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-csg5"
server_key: "3ea1718717f6564977edadb531a3ac8f"
server_key_iv: "68ef278f0b94a84cd2c81375d01fe58e"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.436101 23276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.463370 23276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.468834 23276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:09.469213 23276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.470498 23276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:09.472213 23276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:09.472342 23276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.472517 23276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:09.472600 23276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.515106 23276 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:42151
I20260807 22:43:09.515161 23410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:42151 every 8 connection(s)
I20260807 22:43:09.516654 23276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:09.525061 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23276
I20260807 22:43:09.525323 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:09.526003 20522 external_mini_cluster.cc:1428] Setting key 148b5bad3ddc7c635dc7879f1b8986a5
Aug 07 22:43:09 dist-test-slave-csg5 krb5kdc[23159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:09.544139 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.519997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:41403 (local address 127.20.10.190:46059)
0807 22:43:09.520667 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.520676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.521635 (+ 959us) server_negotiation.cc:407] Connection header received
0807 22:43:09.523761 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.523767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.523881 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.524003 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.525889 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.526785 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.529123 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.529556 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.529683 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.535972 (+ 6289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.535982 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.535991 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.536054 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.539723 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.541067 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.541073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.541075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.541227 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.541981 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.541985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.541988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.542393 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.542527 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.543627 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.543851 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":421,"threads_started":1}
I20260807 22:43:09.545851 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.520573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46059 (local address 127.20.10.129:41403)
0807 22:43:09.521457 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.521503 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.523468 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.524264 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.524272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.524939 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.525699 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.525719 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.527026 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.527034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.528889 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.528909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.530045 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.530057 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.531111 (+ 1054us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.532129 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.532174 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.535576 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.539969 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.539984 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.540009 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.540817 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.541502 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.541510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.541517 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.541834 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.542548 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.542560 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.543122 (+ 562us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.543515 (+ 393us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.544202 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":327,"threads_started":1}
I20260807 22:43:09.547948 23411 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46059
I20260807 22:43:09.548523 23411 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.549438 23411 heartbeater.cc:511] Master 127.20.10.190:46059 requested a full tablet report, sending...
I20260807 22:43:09.552384 23211 ts_manager.cc:194] Registered new tserver with Master: 176504ac4ba340f08947600166b3ef93 (127.20.10.129:42151)
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:43:09.554279 23211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:41403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:43:09.587308 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:09.963956 23417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.964344 23417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.964529 23417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.972302 23417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.972494 23417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.972700 23417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:09.983137 23417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:09.984633 23417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.986084 23417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:10.000705 23423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:10.000743 23424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:10.000769 23426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:10.002465 23417 server_base.cc:1034] running on GCE node
I20260807 22:43:10.003537 23417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:10.005304 23417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:10.006620 23417 hybrid_clock.cc:648] HybridClock initialized: now 1786142590006527 us; error 103 us; skew 500 ppm
I20260807 22:43:10.007102 23417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:10 dist-test-slave-csg5 krb5kdc[23159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142590, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:10.012514 23417 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:10.016057 23417 webserver.cc:460] Webserver started at http://127.20.10.130:37859/ using document root <none> and password file <none>
I20260807 22:43:10.017179 23417 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:10.017418 23417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:10.018236 23417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:10.022603 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "676137d8034d4c3eb6f91ef6f6acc157"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "ad031324f93f5e59380b5c7d5e6e7e3a"
server_key_iv: "157c0a3085dc4692c5c8e05f0c7c8c38"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.024019 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "676137d8034d4c3eb6f91ef6f6acc157"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "ad031324f93f5e59380b5c7d5e6e7e3a"
server_key_iv: "157c0a3085dc4692c5c8e05f0c7c8c38"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.033404 23417 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260807 22:43:10.038048 23433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.039865 23417 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:43:10.040165 23417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "676137d8034d4c3eb6f91ef6f6acc157"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "ad031324f93f5e59380b5c7d5e6e7e3a"
server_key_iv: "157c0a3085dc4692c5c8e05f0c7c8c38"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.040508 23417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:10.058849 23417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:10.064388 23417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:10.064749 23417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:10.065886 23417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:10.067428 23417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:10.067541 23417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.067713 23417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:10.067788 23417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.110532 23417 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:40815
I20260807 22:43:10.110633 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:40815 every 8 connection(s)
I20260807 22:43:10.112008 23417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:10.114444 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23417
I20260807 22:43:10.114642 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:10.115339 20522 external_mini_cluster.cc:1428] Setting key 8729390ed31574731221765774445410
Aug 07 22:43:10 dist-test-slave-csg5 krb5kdc[23159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142590, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:43:10.144055 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.115335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:60933 (local address 127.20.10.190:46059)
0807 22:43:10.123087 (+ 7752us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.123091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.123108 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:10.123165 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.123168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.123253 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.123345 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.125806 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.127202 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.128982 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.129908 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.130076 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.134680 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.134691 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.134700 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.134759 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.139549 (+ 4790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.140770 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.140780 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.140787 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.140975 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.141738 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.141742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.141745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.142077 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.142213 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.143526 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.143759 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7634,"thread_start_us":7523,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
I20260807 22:43:10.145361 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.116002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46059 (local address 127.20.10.130:60933)
0807 22:43:10.117113 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.117189 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.120710 (+ 3521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.123719 (+ 3009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.123735 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.124631 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:10.125538 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.125562 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.127393 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.127398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.128802 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.128817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.130166 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.130171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.130398 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.131344 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:10.131369 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:10.134311 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:10.139853 (+ 5542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.139870 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.139887 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.140576 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.141257 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.141262 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.141265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.141579 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.142449 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:10.142456 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:10.142987 (+ 531us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.143392 (+ 405us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.143886 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":411,"threads_started":1}
I20260807 22:43:10.147318 23552 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46059
I20260807 22:43:10.147738 23552 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:10.148734 23552 heartbeater.cc:511] Master 127.20.10.190:46059 requested a full tablet report, sending...
I20260807 22:43:10.150563 23211 ts_manager.cc:194] Registered new tserver with Master: 676137d8034d4c3eb6f91ef6f6acc157 (127.20.10.130:40815)
I20260807 22:43:10.151544 23211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:60933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:10.180510 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:10.553449 23558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:10.553820 23558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:10.553980 23558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:43:10.558794 23411 heartbeater.cc:503] Master 127.20.10.190:46059 was elected leader, sending a full tablet report...
W20260807 22:43:10.561968 23558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:10.562134 23558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:10.562309 23558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:10.572827 23558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:10.574342 23558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:10.576155 23558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:10.591552 23563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:10.591526 23564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:10.591495 23566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:10.592442 23558 server_base.cc:1034] running on GCE node
I20260807 22:43:10.593201 23558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:10.594379 23558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:10.595669 23558 hybrid_clock.cc:648] HybridClock initialized: now 1786142590595565 us; error 81 us; skew 500 ppm
I20260807 22:43:10.596097 23558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:10 dist-test-slave-csg5 krb5kdc[23159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142590, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:10.600600 23558 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:10.603086 23558 webserver.cc:460] Webserver started at http://127.20.10.131:44363/ using document root <none> and password file <none>
I20260807 22:43:10.604056 23558 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:10.604228 23558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:10.604622 23558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:10.608007 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "6ada1721d08241d2b1136ebc52638759"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "07b896bd0daa44567b5308b4b73f32d2"
server_key_iv: "3cd2dbf659a0e4fee1fec443680beff5"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.608899 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "6ada1721d08241d2b1136ebc52638759"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "07b896bd0daa44567b5308b4b73f32d2"
server_key_iv: "3cd2dbf659a0e4fee1fec443680beff5"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.615263 23558 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:43:10.619779 23573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.621433 23558 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:43:10.621637 23558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "6ada1721d08241d2b1136ebc52638759"
format_stamp: "Formatted at 2026-08-07 22:43:10 on dist-test-slave-csg5"
server_key: "07b896bd0daa44567b5308b4b73f32d2"
server_key_iv: "3cd2dbf659a0e4fee1fec443680beff5"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.621915 23558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:10.639644 23558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:10.644469 23558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:10.644847 23558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:10.646195 23558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:10.647946 23558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:10.648052 23558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.648207 23558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:10.648288 23558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.686512 23558 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:34457
I20260807 22:43:10.686564 23695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:34457 every 8 connection(s)
I20260807 22:43:10.688006 23558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:10.696628 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23558
I20260807 22:43:10.696822 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:10.697288 20522 external_mini_cluster.cc:1428] Setting key 2d92bc9727806e7c5179229e9d1518f8
Aug 07 22:43:10 dist-test-slave-csg5 krb5kdc[23159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142590, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:10.713490 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.691171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:45109 (local address 127.20.10.190:46059)
0807 22:43:10.691728 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.691733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.692824 (+ 1091us) server_negotiation.cc:407] Connection header received
0807 22:43:10.694955 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.694961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.695085 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.695224 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.697214 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.698205 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.700216 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.700823 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.700993 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.705426 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.705432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.705435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.705476 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.709284 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.710352 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.710358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.710361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.710511 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.711156 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.711163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.711165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.711533 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.711727 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.712786 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.713050 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":293,"threads_started":1}
I20260807 22:43:10.714735 23699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.691728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46059 (local address 127.20.10.131:45109)
0807 22:43:10.692608 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.692672 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.694561 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.695514 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.695524 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.696220 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:10.696987 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.697010 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.698397 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.698405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.700042 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.700057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.701110 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.701120 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.701402 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.702243 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:10.702270 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:10.705122 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:10.709522 (+ 4400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.709532 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.709549 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.710100 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.710635 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.710639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.710641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.710990 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.711729 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:10.711740 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:10.712174 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.712571 (+ 397us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.713201 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":327,"threads_started":1}
I20260807 22:43:10.716785 23696 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46059
I20260807 22:43:10.717253 23696 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:10.718410 23696 heartbeater.cc:511] Master 127.20.10.190:46059 requested a full tablet report, sending...
I20260807 22:43:10.720322 23211 ts_manager.cc:194] Registered new tserver with Master: 6ada1721d08241d2b1136ebc52638759 (127.20.10.131:34457)
I20260807 22:43:10.721406 23211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:45109
I20260807 22:43:10.726984 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:43:10.770040 23698 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:10.766649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.20.10.190:46059)
0807 22:43:10.766899 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.766909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.767635 (+ 726us) server_negotiation.cc:407] Connection header received
0807 22:43:10.767836 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.767840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.767936 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.768033 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.769723 (+ 1690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44426: BlockingRecv error: recv got EOF from 127.0.0.1:44426 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:43:10.774380 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23276
W20260807 22:43:10.778084 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23276: Not authorized: Client connection negotiation failed: client connection to 127.20.10.129:42151: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:43:10.778332 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23276
I20260807 22:43:10.798393 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23417
W20260807 22:43:10.801501 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23417: Not authorized: Client connection negotiation failed: client connection to 127.20.10.130:40815: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:43:10.801640 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23417
I20260807 22:43:10.823089 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23558
W20260807 22:43:10.826599 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23558: Not authorized: Client connection negotiation failed: client connection to 127.20.10.131:34457: server requires authentication, but client does not have Kerberos credentials available
I20260807 22:43:10.826849 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23558
I20260807 22:43:10.849416 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23174
I20260807 22:43:10.850922 23243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:10.962489 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23174
2026-08-07T22:43:10Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4208 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:11 dist-test-slave-csg5 krb5kdc[23714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:11 dist-test-slave-csg5 krb5kdc[23714](info): set up 2 sockets
Aug 07 22:43:11 dist-test-slave-csg5 krb5kdc[23714](info): commencing operation
krb5kdc: starting...
W20260807 22:43:12.197010 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:12 dist-test-slave-csg5 krb5kdc[23714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:12.463953 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39525
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:39525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:12.823212 23727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:12.823563 23727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:12.823743 23727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:12.831552 23727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:12.831683 23727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:12.831733 23727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:12.831861 23727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:12.831939 23727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:12.842320 23727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:39525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:12.843838 23727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:12.845242 23727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:12.859452 23736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:12.859452 23733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:12.859458 23734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:12.860363 23727 server_base.cc:1034] running on GCE node
I20260807 22:43:12.861234 23727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:12.862946 23727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:12.864184 23727 hybrid_clock.cc:648] HybridClock initialized: now 1786142592864120 us; error 63 us; skew 500 ppm
I20260807 22:43:12.864526 23727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:12 dist-test-slave-csg5 krb5kdc[23714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142592, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:12.873737 23727 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:12.875586 23727 webserver.cc:460] Webserver started at http://127.20.10.190:41721/ using document root <none> and password file <none>
I20260807 22:43:12.876329 23727 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:12.876446 23727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:12.876736 23727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:12.880352 23727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "0761e562809b4ee0b87092498d66de75"
format_stamp: "Formatted at 2026-08-07 22:43:12 on dist-test-slave-csg5"
server_key: "9d526ea691cc936a8fabf4b1e53e92dd"
server_key_iv: "7f26cc34789fc022fdc9ffbce7689c3a"
server_key_version: "encryptionkey@0"
I20260807 22:43:12.881273 23727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "0761e562809b4ee0b87092498d66de75"
format_stamp: "Formatted at 2026-08-07 22:43:12 on dist-test-slave-csg5"
server_key: "9d526ea691cc936a8fabf4b1e53e92dd"
server_key_iv: "7f26cc34789fc022fdc9ffbce7689c3a"
server_key_version: "encryptionkey@0"
I20260807 22:43:12.887948 23727 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:43:12.893437 23743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:12.895797 23727 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 22:43:12.896057 23727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "0761e562809b4ee0b87092498d66de75"
format_stamp: "Formatted at 2026-08-07 22:43:12 on dist-test-slave-csg5"
server_key: "9d526ea691cc936a8fabf4b1e53e92dd"
server_key_iv: "7f26cc34789fc022fdc9ffbce7689c3a"
server_key_version: "encryptionkey@0"
I20260807 22:43:12.896406 23727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:12.937307 23727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:12.942870 23727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:12.943228 23727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:12.966239 23727 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:39525
I20260807 22:43:12.966353 23801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:39525 every 8 connection(s)
I20260807 22:43:12.967839 23727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:12.970918 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23727
I20260807 22:43:12.971216 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:12.972045 20522 external_mini_cluster.cc:1428] Setting key b778448cbbe6b940a581de9bcf14b8f7
I20260807 22:43:12.972855 23802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:43:12 dist-test-slave-csg5 krb5kdc[23714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:12.988919 23802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75: Bootstrap starting.
I20260807 22:43:12.993531 23802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:12.994982 23802 log.cc:824] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:12.998888 23802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75: No bootstrap required, opened a new log
I20260807 22:43:12.999080 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:12.974224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.20.10.190:39525)
0807 22:43:12.975222 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:12.975240 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:12.975298 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:43:12.977163 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:12.977193 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:12.977791 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:12.978276 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:12.979941 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:12.982105 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:12.983756 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:12.984903 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:12.985126 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:12.989640 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:12.989651 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:12.989671 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:12.989729 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:12.994040 (+ 4311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:12.995012 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:12.995021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:12.995030 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:12.995253 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:12.995747 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:12.995752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:12.995755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:12.996382 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:12.996726 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:12.997133 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 22:43:12.997613 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":429,"threads_started":1}
I20260807 22:43:13.003806 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:13.007952 23802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } }
I20260807 22:43:13.008391 23802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:13.008525 23802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761e562809b4ee0b87092498d66de75, State: Initialized, Role: FOLLOWER
I20260807 22:43:13.009234 23802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } }
I20260807 22:43:13.009511 23802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:13.009652 23802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:13.009850 23802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:13.012586 23802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } }
I20260807 22:43:13.013315 23802 leader_election.cc:302] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0761e562809b4ee0b87092498d66de75; no voters:
I20260807 22:43:13.014144 23802 leader_election.cc:288] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:13.014307 23808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:13.015652 23808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [term 1 LEADER]: Becoming Leader. State: Replica: 0761e562809b4ee0b87092498d66de75, State: Running, Role: LEADER
I20260807 22:43:13.016428 23808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } }
I20260807 22:43:13.016649 23802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:13.020638 23809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0761e562809b4ee0b87092498d66de75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } } }
I20260807 22:43:13.021126 23809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:13.021786 23810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0761e562809b4ee0b87092498d66de75. Latest consensus state: current_term: 1 leader_uuid: "0761e562809b4ee0b87092498d66de75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0761e562809b4ee0b87092498d66de75" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39525 } } }
I20260807 22:43:13.022133 23810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:13.029557 23816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:13.035809 23816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:13.048105 23816 catalog_manager.cc:1347] Generated new cluster ID: dfa11c13b4014b87aa90b103e8389739
I20260807 22:43:13.048246 23816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:13.071329 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.060658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.20.10.190:39525)
0807 22:43:13.060872 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.060880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.061566 (+ 686us) server_negotiation.cc:407] Connection header received
0807 22:43:13.061782 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.061787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.061876 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.061978 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.063527 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.064636 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.066739 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.067630 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.067802 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.069124 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.069130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.069134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:13.070226 (+ 1092us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.070289 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:13.070317 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.070457 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.070625 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.070903 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:43:13.071523 23816 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:13.073156 23816 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:13.081278 23816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0761e562809b4ee0b87092498d66de75: Generated new TSK 0
I20260807 22:43:13.082453 23816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:13.095078 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23727
I20260807 22:43:13.096657 23796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:13.198166 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23727
2026-08-07T22:43:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2230 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:13 dist-test-slave-csg5 krb5kdc[23837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:13 dist-test-slave-csg5 krb5kdc[23837](info): set up 2 sockets
Aug 07 22:43:13 dist-test-slave-csg5 krb5kdc[23837](info): commencing operation
krb5kdc: starting...
W20260807 22:43:14.383450 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:14 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:14.656693 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46535
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:15.036441 23850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:15.036800 23850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:15.036957 23850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:15.045249 23850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:15.045383 23850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:15.045475 23850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:15.045594 23850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:15.045676 23850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:15.057430 23850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:15.058949 23850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:15.060359 23850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:15.073946 23856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:15.073908 23857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:15.073872 23860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:15.074342 23850 server_base.cc:1034] running on GCE node
I20260807 22:43:15.075529 23850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:15.077250 23850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:15.078529 23850 hybrid_clock.cc:648] HybridClock initialized: now 1786142595078477 us; error 37 us; skew 500 ppm
I20260807 22:43:15.079008 23850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:15 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142595, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:15.082806 23850 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:15.084834 23850 webserver.cc:460] Webserver started at http://127.20.10.190:33441/ using document root <none> and password file <none>
I20260807 22:43:15.085901 23850 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:15.086050 23850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:15.086333 23850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:15.089705 23850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "33535f29be6643f1abe55207aba8678b"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5d4f7b82b8a8d1aa2cf9a2254d4307b8"
server_key_iv: "c995cc1bfa3384788bcc86ef4c452ae2"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.090534 23850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "33535f29be6643f1abe55207aba8678b"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5d4f7b82b8a8d1aa2cf9a2254d4307b8"
server_key_iv: "c995cc1bfa3384788bcc86ef4c452ae2"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.096513 23850 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 22:43:15.100548 23867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:15.102264 23850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:43:15.102478 23850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "33535f29be6643f1abe55207aba8678b"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5d4f7b82b8a8d1aa2cf9a2254d4307b8"
server_key_iv: "c995cc1bfa3384788bcc86ef4c452ae2"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.102843 23850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:15.126629 23850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:15.132064 23850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:15.132349 23850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:15.153848 23850 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46535
I20260807 22:43:15.153880 23920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46535 every 8 connection(s)
I20260807 22:43:15.155424 23850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:15.156991 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23850
I20260807 22:43:15.157238 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:15.158066 20522 external_mini_cluster.cc:1428] Setting key 776551a89282fb8006d3880f67692d92
I20260807 22:43:15.160691 23921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:43:15 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:15.176537 23921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b: Bootstrap starting.
I20260807 22:43:15.180562 23921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:15.181797 23921 log.cc:824] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:15.186092 23921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b: No bootstrap required, opened a new log
I20260807 22:43:15.187541 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:15.160475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.20.10.190:46535)
0807 22:43:15.161745 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:15.161758 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:15.161802 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:43:15.163797 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:15.163824 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:15.164369 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:15.164799 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:15.167061 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.169031 (+ 1970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.170994 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.171989 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.172177 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:15.177648 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:15.177659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:15.177677 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:15.177739 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:15.181754 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:15.182978 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:15.182991 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:15.183005 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:15.183290 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:15.183837 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:15.183841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:15.183843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:15.184485 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:15.184913 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:15.185756 (+ 843us) server_negotiation.cc:299] Negotiation successful
0807 22:43:15.186318 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":709,"threads_started":1}
I20260807 22:43:15.194535 23921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } }
I20260807 22:43:15.195078 23921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:15.195219 23921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33535f29be6643f1abe55207aba8678b, State: Initialized, Role: FOLLOWER
I20260807 22:43:15.195926 23921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } }
I20260807 22:43:15.196219 23921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:15.196400 23921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:15.196612 23921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:15.199126 23921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } }
I20260807 22:43:15.199692 23921 leader_election.cc:302] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33535f29be6643f1abe55207aba8678b; no voters:
I20260807 22:43:15.200445 23921 leader_election.cc:288] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:15.200562 23926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:15.202147 23926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [term 1 LEADER]: Becoming Leader. State: Replica: 33535f29be6643f1abe55207aba8678b, State: Running, Role: LEADER
I20260807 22:43:15.202916 23926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } }
I20260807 22:43:15.203275 23921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:15.206502 23928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33535f29be6643f1abe55207aba8678b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } } }
I20260807 22:43:15.207031 23928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:15.207270 23929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33535f29be6643f1abe55207aba8678b. Latest consensus state: current_term: 1 leader_uuid: "33535f29be6643f1abe55207aba8678b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33535f29be6643f1abe55207aba8678b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46535 } } }
I20260807 22:43:15.207705 23929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:15.213896 23934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:15.219972 23934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:15.231627 23934 catalog_manager.cc:1347] Generated new cluster ID: 5ca989a685ef42d49960d276d7956b7d
I20260807 22:43:15.231803 23934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:15.242381 23934 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:15.243899 23934 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:15.251372 23934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33535f29be6643f1abe55207aba8678b: Generated new TSK 0
I20260807 22:43:15.252468 23934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:43:15.335649 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:15.717512 23950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:15.717859 23950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:15.718075 23950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:15.726255 23950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:15.726397 23950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:15.726599 23950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:15.737761 23950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:15.739403 23950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:15.740911 23950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:15.757040 23956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:15.757130 23958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:15.757372 23960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:15.758463 23950 server_base.cc:1034] running on GCE node
I20260807 22:43:15.759243 23950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:15.760322 23950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:15.761574 23950 hybrid_clock.cc:648] HybridClock initialized: now 1786142595761495 us; error 64 us; skew 500 ppm
I20260807 22:43:15.761976 23950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:15 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142595, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:15.766319 23950 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:43:15.768312 23950 webserver.cc:460] Webserver started at http://127.20.10.129:36057/ using document root <none> and password file <none>
I20260807 22:43:15.769155 23950 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:15.769287 23950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:15.769579 23950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:15.773156 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "7e91befa32b34c7b924b92e29228dbd4"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5b66947a9d2943968b5860247b0e263c"
server_key_iv: "6cb38ab9a4f5d16b4a02decc6438205d"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.774245 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "7e91befa32b34c7b924b92e29228dbd4"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5b66947a9d2943968b5860247b0e263c"
server_key_iv: "6cb38ab9a4f5d16b4a02decc6438205d"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.780943 23950 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 22:43:15.785395 23967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:15.787168 23950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:15.787367 23950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "7e91befa32b34c7b924b92e29228dbd4"
format_stamp: "Formatted at 2026-08-07 22:43:15 on dist-test-slave-csg5"
server_key: "5b66947a9d2943968b5860247b0e263c"
server_key_iv: "6cb38ab9a4f5d16b4a02decc6438205d"
server_key_version: "encryptionkey@0"
I20260807 22:43:15.787590 23950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:15.804021 23950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:15.809345 23950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:15.809643 23950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:15.810732 23950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:15.812247 23950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:15.812326 23950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:15.812422 23950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:15.812472 23950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:15.861274 23950 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:37327
I20260807 22:43:15.861286 24095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:37327 every 8 connection(s)
I20260807 22:43:15.863401 23950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:15.871206 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 23950
I20260807 22:43:15.871474 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:15.872149 20522 external_mini_cluster.cc:1428] Setting key 714cbe50b70369bca1724a0e51240c16
Aug 07 22:43:15 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142595, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:43:15.905066 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:15.876696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:59309 (local address 127.20.10.190:46535)
0807 22:43:15.877288 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:15.877295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:15.881728 (+ 4433us) server_negotiation.cc:407] Connection header received
0807 22:43:15.881825 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:15.881834 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:15.881947 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:15.882079 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:15.884747 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.885624 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.888548 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.889391 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.889565 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:15.895679 (+ 6114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:15.895690 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:15.895697 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:15.895759 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:15.900314 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:15.901599 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:15.901604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:15.901607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:15.901866 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:15.902462 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:15.902468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:15.902472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:15.902889 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:15.903075 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:15.904222 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0807 22:43:15.904579 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":254,"threads_started":1}
I20260807 22:43:15.906224 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:15.876628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46535 (local address 127.20.10.129:59309)
0807 22:43:15.877789 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:15.877863 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:15.881217 (+ 3354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:15.882455 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:15.882473 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:15.883471 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:15.884452 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:15.884484 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.885872 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.885879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:15.888314 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:15.888334 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:15.889710 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:15.889720 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:15.890113 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:15.891185 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:15.891226 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:15.895225 (+ 3999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:15.900583 (+ 5358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:15.900598 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:15.900620 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:15.901402 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:15.902021 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:15.902027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:15.902031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:15.902309 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:15.903051 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:15.903060 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:15.903615 (+ 555us) client_negotiation.cc:769] Sending connection context
0807 22:43:15.904070 (+ 455us) client_negotiation.cc:240] Negotiation successful
0807 22:43:15.904745 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":478,"threads_started":1}
I20260807 22:43:15.908318 24096 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46535
I20260807 22:43:15.908715 24096 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:15.909619 24096 heartbeater.cc:511] Master 127.20.10.190:46535 requested a full tablet report, sending...
I20260807 22:43:15.912353 23884 ts_manager.cc:194] Registered new tserver with Master: 7e91befa32b34c7b924b92e29228dbd4 (127.20.10.129:37327)
I20260807 22:43:15.914270 23884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:59309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:43:15.940111 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:16.325856 24104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:16.326196 24104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:16.326313 24104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:16.333962 24104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:16.334101 24104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:16.334223 24104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:16.344492 24104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:16.345913 24104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:16.347335 24104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:16.366515 24111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.366539 24113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.366912 24110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:16.367326 24104 server_base.cc:1034] running on GCE node
I20260807 22:43:16.368077 24104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:16.369378 24104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:16.370707 24104 hybrid_clock.cc:648] HybridClock initialized: now 1786142596370617 us; error 30 us; skew 500 ppm
I20260807 22:43:16.371124 24104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:16 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142596, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:16.374927 24104 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:16.376715 24104 webserver.cc:460] Webserver started at http://127.20.10.130:41225/ using document root <none> and password file <none>
I20260807 22:43:16.377491 24104 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:16.377657 24104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:16.378005 24104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:16.381692 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "2c7419ef0c394d5fac8fe71f0ba2e510"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "836bd047b1bb10895601dc5ac325b850"
server_key_iv: "8bae10ba0354d94cc0006debedee5421"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.382596 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "2c7419ef0c394d5fac8fe71f0ba2e510"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "836bd047b1bb10895601dc5ac325b850"
server_key_iv: "8bae10ba0354d94cc0006debedee5421"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.388928 24104 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:16.393348 24121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:16.395249 24104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:16.395532 24104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "2c7419ef0c394d5fac8fe71f0ba2e510"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "836bd047b1bb10895601dc5ac325b850"
server_key_iv: "8bae10ba0354d94cc0006debedee5421"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.395774 24104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:16.424777 24104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:16.429951 24104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:16.430318 24104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:16.431589 24104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:16.433192 24104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:16.433316 24104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:16.433475 24104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:16.433552 24104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:16.473227 24104 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:42625
I20260807 22:43:16.473351 24243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:42625 every 8 connection(s)
I20260807 22:43:16.474781 24104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:16.478825 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24104
I20260807 22:43:16.479069 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:16.479696 20522 external_mini_cluster.cc:1428] Setting key a941fa6d9b913aa37c2bf670e90f927a
Aug 07 22:43:16 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142596, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:16.500231 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:16.477986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:43033 (local address 127.20.10.190:46535)
0807 22:43:16.478564 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:16.478570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:16.479763 (+ 1193us) server_negotiation.cc:407] Connection header received
0807 22:43:16.481995 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:16.482001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:16.482106 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:16.482248 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:16.484354 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:16.485581 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:16.487653 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:16.488440 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:16.488592 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:16.492863 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:16.492868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:16.492871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:16.492909 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:16.496175 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:16.497101 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:16.497106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:16.497108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:16.497251 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:16.497759 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:16.497771 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:16.497773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:16.498094 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:16.498279 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:16.499286 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0807 22:43:16.499634 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":268,"threads_started":1}
I20260807 22:43:16.501412 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:16.478587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46535 (local address 127.20.10.130:43033)
0807 22:43:16.479546 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:16.479615 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:16.481569 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:16.482570 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:16.482586 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:16.483369 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:16.484039 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:16.484059 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:16.485781 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:16.485786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:16.487465 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:16.487492 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:16.488714 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:16.488720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:16.488955 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:16.489829 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:16.489855 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:16.492534 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:16.496361 (+ 3827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:16.496372 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:16.496393 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:16.496937 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:16.497382 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:16.497387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:16.497391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:16.497618 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:16.498251 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:16.498260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:16.498744 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 22:43:16.499084 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 22:43:16.499655 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":374,"threads_started":1}
I20260807 22:43:16.503197 24244 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46535
I20260807 22:43:16.503638 24244 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:43:16.504803 24244 heartbeater.cc:511] Master 127.20.10.190:46535 requested a full tablet report, sending...
I20260807 22:43:16.506942 23884 ts_manager.cc:194] Registered new tserver with Master: 2c7419ef0c394d5fac8fe71f0ba2e510 (127.20.10.130:42625)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
I20260807 22:43:16.508201 23884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:43033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:16.543577 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:16.911954 24250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:16.912410 24250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:16.912601 24250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:43:16.919039 24096 heartbeater.cc:503] Master 127.20.10.190:46535 was elected leader, sending a full tablet report...
W20260807 22:43:16.921840 24250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:16.922026 24250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:16.922339 24250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:16.933791 24250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:16.935442 24250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:16.936937 24250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:16.952134 24257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.952092 24259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.952390 24256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:16.953599 24250 server_base.cc:1034] running on GCE node
I20260807 22:43:16.954384 24250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:16.955597 24250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:16.956830 24250 hybrid_clock.cc:648] HybridClock initialized: now 1786142596956791 us; error 32 us; skew 500 ppm
I20260807 22:43:16.957223 24250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:16 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142596, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:16.960994 24250 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:16.962591 24250 webserver.cc:460] Webserver started at http://127.20.10.131:45227/ using document root <none> and password file <none>
I20260807 22:43:16.963382 24250 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:16.963495 24250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:16.963754 24250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:16.967060 24250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "93f3a3f311f44ab792b74da2abb1e6fc"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "fdec873b5c8d5aceb8fa231949022735"
server_key_iv: "da2a3466a92d5d3f511f7b29debbcd4a"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.967887 24250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "93f3a3f311f44ab792b74da2abb1e6fc"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "fdec873b5c8d5aceb8fa231949022735"
server_key_iv: "da2a3466a92d5d3f511f7b29debbcd4a"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.973893 24250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:43:16.978018 24266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:16.979733 24250 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:16.979889 24250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "93f3a3f311f44ab792b74da2abb1e6fc"
format_stamp: "Formatted at 2026-08-07 22:43:16 on dist-test-slave-csg5"
server_key: "fdec873b5c8d5aceb8fa231949022735"
server_key_iv: "da2a3466a92d5d3f511f7b29debbcd4a"
server_key_version: "encryptionkey@0"
I20260807 22:43:16.980074 24250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:17.022264 24250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:17.027148 24250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:17.027446 24250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:17.028434 24250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:17.029958 24250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:17.030035 24250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:17.030179 24250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:17.030241 24250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:17.068583 24250 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:46351
I20260807 22:43:17.068699 24391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:46351 every 8 connection(s)
I20260807 22:43:17.070050 24250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:17.072388 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24250
I20260807 22:43:17.072762 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:17.073436 20522 external_mini_cluster.cc:1428] Setting key d7c6ad1176a770e492d0093363280d1f
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142596, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:17.095479 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.073722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:36197 (local address 127.20.10.190:46535)
0807 22:43:17.074295 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.074302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.075664 (+ 1362us) server_negotiation.cc:407] Connection header received
0807 22:43:17.077771 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.077775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.077857 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.077955 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.079991 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.080857 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.082718 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.083260 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.083374 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.088139 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.088145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.088149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.088189 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.091969 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.093054 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.093059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.093061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.093196 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.093711 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.093716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.093719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.094013 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.094136 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.094988 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.095171 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":277,"threads_started":1}
I20260807 22:43:17.096634 24395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.074439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46535 (local address 127.20.10.131:36197)
0807 22:43:17.075478 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:17.075530 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:17.077468 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:17.078272 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:17.078282 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:17.079059 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:17.079778 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:17.079800 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.081025 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.081029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:17.082475 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:17.082491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.083566 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.083573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:17.083838 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.084754 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:17.084782 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:17.087829 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:17.092161 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:17.092180 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:17.092197 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:17.092902 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:17.093378 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:17.093382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:17.093384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:17.093603 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:17.094142 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:17.094150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:17.094530 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 22:43:17.094881 (+ 351us) client_negotiation.cc:240] Negotiation successful
0807 22:43:17.095333 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":532,"threads_started":1}
I20260807 22:43:17.098403 24392 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46535
I20260807 22:43:17.098820 24392 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:17.099859 24392 heartbeater.cc:511] Master 127.20.10.190:46535 requested a full tablet report, sending...
I20260807 22:43:17.101930 23883 ts_manager.cc:194] Registered new tserver with Master: 93f3a3f311f44ab792b74da2abb1e6fc (127.20.10.131:46351)
I20260807 22:43:17.103411 23883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:36197
I20260807 22:43:17.103498 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:17.168941 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.150806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.20.10.190:46535)
0807 22:43:17.151130 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.151140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.151554 (+ 414us) server_negotiation.cc:407] Connection header received
0807 22:43:17.151840 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.151852 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.151929 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.152056 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.153968 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.154874 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.156944 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.157435 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.157554 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.162844 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.162850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.162853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.162892 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.165980 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.166776 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.166784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.166794 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.166929 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.167362 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.167366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.167368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.167707 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.167994 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.168305 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.168572 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
W20260807 22:43:17.169693 23883 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46768
I20260807 22:43:17.171491 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23950
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:17.189750 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.172635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.20.10.129:37327)
0807 22:43:17.173409 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.173417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.173445 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:43:17.173512 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.173519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.173750 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.173896 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.175436 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.176525 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.177801 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.178526 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.178648 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.183205 (+ 4557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.183222 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.183239 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.183284 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.186629 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.187462 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.187470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.187473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.187625 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.188070 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.188074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.188076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.188469 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.188779 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.189143 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.189382 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":354,"threads_started":1}
W20260807 22:43:17.191407 24091 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57234
W20260807 22:43:17.192884 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23950: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:17.193099 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23950
I20260807 22:43:17.217805 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24104
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:43:17.237390 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.219006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.20.10.130:42625)
0807 22:43:17.219663 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.219670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.219697 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:17.219759 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.219766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.219978 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.220110 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.221495 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.222915 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.224345 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.224963 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.225102 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.230206 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.230228 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.230245 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.230293 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.233841 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.234797 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.234806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.234808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.234966 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.235525 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.235530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.235533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.236001 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.236315 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.236707 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.236970 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":346,"threads_started":1}
W20260807 22:43:17.238922 24239 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59424
W20260807 22:43:17.240175 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24104: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:17.240502 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24104
I20260807 22:43:17.264410 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24250
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[23837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:43:17.284704 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.265626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.20.10.131:46351)
0807 22:43:17.266473 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.266480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.266510 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:43:17.266572 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.266578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.266869 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.267006 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.268885 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.270055 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.271750 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.272516 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.272672 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.277970 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.277990 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.278006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.278051 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.281323 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.282232 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.282241 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.282243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.282389 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.282929 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.282934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.282936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.283410 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.283864 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.284057 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.284289 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":509,"threads_started":1}
W20260807 22:43:17.286142 24386 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46202
W20260807 22:43:17.287948 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24250: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:43:17.288120 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24250
I20260807 22:43:17.309653 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 23850
I20260807 22:43:17.311036 23916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:17.425009 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 23850
2026-08-07T22:43:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4232 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[24411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[24411](info): set up 2 sockets
Aug 07 22:43:17 dist-test-slave-csg5 krb5kdc[24411](info): commencing operation
krb5kdc: starting...
W20260807 22:43:18.648824 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:18 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:18.912104 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:38009
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:38009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:19.281226 24424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:19.281587 24424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:19.281687 24424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:19.289381 24424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:19.289516 24424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:19.289566 24424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:19.289620 24424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:19.289667 24424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:19.300665 24424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:38009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:38009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:19.302207 24424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:19.303720 24424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:19.316983 24430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.316957 24431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.316918 24433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.318159 24424 server_base.cc:1034] running on GCE node
I20260807 22:43:19.319101 24424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:19.320670 24424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:19.321914 24424 hybrid_clock.cc:648] HybridClock initialized: now 1786142599321850 us; error 58 us; skew 500 ppm
I20260807 22:43:19.322311 24424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:19 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142599, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:19.326068 24424 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:19.328073 24424 webserver.cc:460] Webserver started at http://127.20.10.190:45173/ using document root <none> and password file <none>
I20260807 22:43:19.328881 24424 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:19.329038 24424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:19.329442 24424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:19.332808 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "00b46cfad9c14c94be02cbe3b236458b"
format_stamp: "Formatted at 2026-08-07 22:43:19 on dist-test-slave-csg5"
server_key: "2880e7e11c438447b8fb843520fdebd6"
server_key_iv: "d7a3a8bd84e95eaca593ee90b4131907"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.333683 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "00b46cfad9c14c94be02cbe3b236458b"
format_stamp: "Formatted at 2026-08-07 22:43:19 on dist-test-slave-csg5"
server_key: "2880e7e11c438447b8fb843520fdebd6"
server_key_iv: "d7a3a8bd84e95eaca593ee90b4131907"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.339694 24424 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 22:43:19.343915 24440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.345461 24424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:43:19.345688 24424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "00b46cfad9c14c94be02cbe3b236458b"
format_stamp: "Formatted at 2026-08-07 22:43:19 on dist-test-slave-csg5"
server_key: "2880e7e11c438447b8fb843520fdebd6"
server_key_iv: "d7a3a8bd84e95eaca593ee90b4131907"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.345964 24424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:19.373169 24424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:19.377943 24424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:19.378311 24424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:19.399206 24424 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:38009
I20260807 22:43:19.399264 24494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:38009 every 8 connection(s)
I20260807 22:43:19.400707 24424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:19.405268 24495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:19.410378 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24424
I20260807 22:43:19.410673 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:19.411643 20522 external_mini_cluster.cc:1428] Setting key 02aacdcb3669ae6d92d1ae1f0ad7c1fc
I20260807 22:43:19.419631 24495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b: Bootstrap starting.
I20260807 22:43:19.424108 24495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:19.425643 24495 log.cc:824] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:19.430107 24495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b: No bootstrap required, opened a new log
Aug 07 22:43:19 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:19.442744 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.414167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.20.10.190:38009)
0807 22:43:19.415059 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.415072 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.415115 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:43:19.417820 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.417848 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.418299 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.418745 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.420914 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.422509 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.424661 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.425507 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.425709 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.431595 (+ 5886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:19.431609 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:19.431633 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:19.431695 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:19.436748 (+ 5053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:19.437747 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:19.437760 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:19.437771 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:19.438053 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:19.438553 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:19.438560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:19.438564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:19.439512 (+ 948us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:19.439884 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:19.440401 (+ 517us) server_negotiation.cc:299] Negotiation successful
0807 22:43:19.440998 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":426,"threads_started":1}
I20260807 22:43:19.443351 24495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } }
I20260807 22:43:19.443823 24495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:19.443960 24495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b46cfad9c14c94be02cbe3b236458b, State: Initialized, Role: FOLLOWER
I20260807 22:43:19.444698 24495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } }
I20260807 22:43:19.445019 24495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:19.445225 24495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:19.445449 24495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:19.448391 24495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } }
I20260807 22:43:19.448982 24495 leader_election.cc:302] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b46cfad9c14c94be02cbe3b236458b; no voters:
I20260807 22:43:19.449779 24495 leader_election.cc:288] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:19.449947 24505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:19.451094 24505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [term 1 LEADER]: Becoming Leader. State: Replica: 00b46cfad9c14c94be02cbe3b236458b, State: Running, Role: LEADER
I20260807 22:43:19.451749 24505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } }
I20260807 22:43:19.452898 24495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:19.455273 24508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b46cfad9c14c94be02cbe3b236458b. Latest consensus state: current_term: 1 leader_uuid: "00b46cfad9c14c94be02cbe3b236458b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } } }
I20260807 22:43:19.455811 24508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:19.456400 24507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b46cfad9c14c94be02cbe3b236458b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b46cfad9c14c94be02cbe3b236458b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38009 } } }
I20260807 22:43:19.456887 24507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:19.460410 24514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:19.469159 24514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:19.481729 24514 catalog_manager.cc:1347] Generated new cluster ID: 93797de484474479b6403bec8b1b3e2e
I20260807 22:43:19.481894 24514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:19.510390 24514 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:19.512794 24514 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:19.528117 24514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b46cfad9c14c94be02cbe3b236458b: Generated new TSK 0
I20260807 22:43:19.529163 24514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:43:19.596432 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:19.968737 24529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:19.969141 24529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:19.969345 24529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:19.977530 24529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:19.977727 24529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:19.977933 24529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:19.988724 24529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:19.990212 24529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:19.991772 24529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:20.007112 24535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:20.007205 24538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:20.007273 24536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:20.009097 24529 server_base.cc:1034] running on GCE node
I20260807 22:43:20.010005 24529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:20.011358 24529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:20.012645 24529 hybrid_clock.cc:648] HybridClock initialized: now 1786142600012579 us; error 57 us; skew 500 ppm
I20260807 22:43:20.013038 24529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:20 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142600, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:20.016952 24529 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:43:20.018992 24529 webserver.cc:460] Webserver started at http://127.20.10.129:39013/ using document root <none> and password file <none>
I20260807 22:43:20.019809 24529 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:20.019981 24529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:20.020347 24529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:20.023744 24529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "225a999b682f47049d05a20cc579142d"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "40ab7e2ea9f66adedc53169db6b16107"
server_key_iv: "9d521fcabc42f766215657ef686378c3"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.024700 24529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "225a999b682f47049d05a20cc579142d"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "40ab7e2ea9f66adedc53169db6b16107"
server_key_iv: "9d521fcabc42f766215657ef686378c3"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.032295 24529 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260807 22:43:20.036757 24545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.038380 24529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:43:20.038614 24529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "225a999b682f47049d05a20cc579142d"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "40ab7e2ea9f66adedc53169db6b16107"
server_key_iv: "9d521fcabc42f766215657ef686378c3"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.038980 24529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:20.077520 24529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:20.082489 24529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:20.083127 24529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:20.084600 24529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:20.086146 24529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:20.086258 24529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.086423 24529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:20.086500 24529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.127703 24529 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:34055
I20260807 22:43:20.127744 24667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:34055 every 8 connection(s)
I20260807 22:43:20.129199 24529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:20.135768 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24529
I20260807 22:43:20.136014 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:20.136942 20522 external_mini_cluster.cc:1428] Setting key 6a81540483dc40f4f6793cb79c9b4b2d
Aug 07 22:43:20 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142600, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:20.157428 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.132511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:56205 (local address 127.20.10.190:38009)
0807 22:43:20.133081 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:20.133088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:20.134203 (+ 1115us) server_negotiation.cc:407] Connection header received
0807 22:43:20.137784 (+ 3581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:20.137791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:20.137902 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:20.138046 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:20.140116 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.140986 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.143858 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.144601 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.144778 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.149848 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:20.149854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:20.149857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:20.149896 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:20.153030 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:20.154155 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:20.154165 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:20.154172 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:20.154377 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:20.155057 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:20.155066 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:20.155071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:20.155505 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:20.155675 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:20.156599 (+ 924us) server_negotiation.cc:299] Negotiation successful
0807 22:43:20.156911 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
I20260807 22:43:20.158635 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.133068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:38009 (local address 127.20.10.129:56205)
0807 22:43:20.133992 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:20.134043 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:20.137372 (+ 3329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:20.138353 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:20.138363 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:20.139058 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:20.139894 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.139924 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.141178 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.141187 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.143608 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.143630 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.144919 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.144926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.145306 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.146195 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:20.146223 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:20.149385 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:20.153214 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:20.153225 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:20.153250 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:20.153938 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:20.154550 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:20.154554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:20.154557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:20.154895 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:20.155684 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:20.155691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:20.156123 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 22:43:20.156409 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 22:43:20.156910 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":394,"threads_started":1}
I20260807 22:43:20.161114 24669 heartbeater.cc:347] Connected to a master server at 127.20.10.190:38009
I20260807 22:43:20.161625 24669 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:20.162515 24669 heartbeater.cc:511] Master 127.20.10.190:38009 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:43:20.165561 24458 ts_manager.cc:194] Registered new tserver with Master: 225a999b682f47049d05a20cc579142d (127.20.10.129:34055)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:43:20.168138 24458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:56205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:43:20.199481 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:20.585119 24676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:20.585460 24676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:20.585650 24676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:20.594074 24676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:20.594231 24676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:20.594502 24676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:20.607303 24676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:20.608826 24676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:20.610333 24676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:20.625857 24681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:20.625754 24684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:20.625785 24682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:20.626827 24676 server_base.cc:1034] running on GCE node
I20260807 22:43:20.627534 24676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:20.628736 24676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:20.630012 24676 hybrid_clock.cc:648] HybridClock initialized: now 1786142600629942 us; error 74 us; skew 500 ppm
I20260807 22:43:20.630369 24676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:20 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142600, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:20.634306 24676 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:20.636332 24676 webserver.cc:460] Webserver started at http://127.20.10.130:45443/ using document root <none> and password file <none>
I20260807 22:43:20.637094 24676 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:20.637251 24676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:20.637590 24676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:20.640841 24676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "6ed0524ff256441182c1dcd6fb45ec92"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "1f7ec79f3988aefb1d94415c1dcf4adb"
server_key_iv: "5e11198e330dd9c0db63fc81978dd7e8"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.641742 24676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "6ed0524ff256441182c1dcd6fb45ec92"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "1f7ec79f3988aefb1d94415c1dcf4adb"
server_key_iv: "5e11198e330dd9c0db63fc81978dd7e8"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.647811 24676 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:43:20.651983 24692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.653636 24676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:43:20.653861 24676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "6ed0524ff256441182c1dcd6fb45ec92"
format_stamp: "Formatted at 2026-08-07 22:43:20 on dist-test-slave-csg5"
server_key: "1f7ec79f3988aefb1d94415c1dcf4adb"
server_key_iv: "5e11198e330dd9c0db63fc81978dd7e8"
server_key_version: "encryptionkey@0"
I20260807 22:43:20.654107 24676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:20.670252 24676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:20.675695 24676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:20.676081 24676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:20.677229 24676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:20.678931 24676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:20.679044 24676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.679205 24676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:20.679287 24676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:20.718708 24676 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35083
I20260807 22:43:20.718745 24814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35083 every 8 connection(s)
I20260807 22:43:20.720176 24676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:20.727370 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24676
I20260807 22:43:20.727571 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:20.728075 20522 external_mini_cluster.cc:1428] Setting key 3554edb513a284d137be6b7637e560f1
Aug 07 22:43:20 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142600, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:20.746281 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.723300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:39259 (local address 127.20.10.190:38009)
0807 22:43:20.723876 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:20.723882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:20.725026 (+ 1144us) server_negotiation.cc:407] Connection header received
0807 22:43:20.727359 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:20.727365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:20.727500 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:20.727643 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:20.729519 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.730402 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.732167 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.732715 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.732859 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.738077 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:20.738083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:20.738087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:20.738124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:20.741665 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:20.742730 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:20.742740 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:20.742747 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:20.742967 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:20.743584 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:20.743593 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:20.743600 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:20.744083 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:20.744290 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:20.745322 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0807 22:43:20.745683 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":302,"threads_started":1}
I20260807 22:43:20.746893 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.724002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:38009 (local address 127.20.10.130:39259)
0807 22:43:20.724830 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:20.724882 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:20.727022 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:20.727929 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:20.727939 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:20.728518 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:20.729328 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.729349 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.730581 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.730586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.731988 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.732002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.733067 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.733078 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.733534 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.734453 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:20.734494 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:20.737753 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:20.741886 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:20.741897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:20.741918 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:20.742441 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:20.743153 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:20.743159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:20.743164 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:20.743424 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:20.744303 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:20.744313 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:20.744810 (+ 497us) client_negotiation.cc:769] Sending connection context
0807 22:43:20.745126 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 22:43:20.745607 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":315,"threads_started":1}
I20260807 22:43:20.748775 24815 heartbeater.cc:347] Connected to a master server at 127.20.10.190:38009
I20260807 22:43:20.749147 24815 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:20.750101 24815 heartbeater.cc:511] Master 127.20.10.190:38009 requested a full tablet report, sending...
I20260807 22:43:20.751994 24458 ts_manager.cc:194] Registered new tserver with Master: 6ed0524ff256441182c1dcd6fb45ec92 (127.20.10.130:35083)
I20260807 22:43:20.753021 24458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:39259
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:20.792421 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:21.167917 24821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:21.168263 24821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:21.168439 24821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:43:21.171945 24669 heartbeater.cc:503] Master 127.20.10.190:38009 was elected leader, sending a full tablet report...
W20260807 22:43:21.177610 24821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:21.177793 24821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:21.177990 24821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:21.189566 24821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:38009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:21.191129 24821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:21.193163 24821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:21.209168 24827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:21.209256 24830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:21.209288 24828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:21.210773 24821 server_base.cc:1034] running on GCE node
I20260807 22:43:21.211524 24821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:21.212671 24821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:21.213935 24821 hybrid_clock.cc:648] HybridClock initialized: now 1786142601213881 us; error 36 us; skew 500 ppm
I20260807 22:43:21.214382 24821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:21.218537 24821 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:21.220427 24821 webserver.cc:460] Webserver started at http://127.20.10.131:41425/ using document root <none> and password file <none>
I20260807 22:43:21.221266 24821 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:21.221436 24821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:21.221810 24821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:21.225375 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "9dc329d90eec47cb81120617b8458233"
format_stamp: "Formatted at 2026-08-07 22:43:21 on dist-test-slave-csg5"
server_key: "de026a920ffd71255009dc457ecd1f24"
server_key_iv: "16218bd36369ec8d31a767407b96a033"
server_key_version: "encryptionkey@0"
I20260807 22:43:21.226361 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "9dc329d90eec47cb81120617b8458233"
format_stamp: "Formatted at 2026-08-07 22:43:21 on dist-test-slave-csg5"
server_key: "de026a920ffd71255009dc457ecd1f24"
server_key_iv: "16218bd36369ec8d31a767407b96a033"
server_key_version: "encryptionkey@0"
I20260807 22:43:21.232689 24821 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 22:43:21.237450 24838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:21.239393 24821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 22:43:21.239637 24821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "9dc329d90eec47cb81120617b8458233"
format_stamp: "Formatted at 2026-08-07 22:43:21 on dist-test-slave-csg5"
server_key: "de026a920ffd71255009dc457ecd1f24"
server_key_iv: "16218bd36369ec8d31a767407b96a033"
server_key_version: "encryptionkey@0"
I20260807 22:43:21.239923 24821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:21.257723 24821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:21.263266 24821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:21.263677 24821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:21.264945 24821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:21.266746 24821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:21.266860 24821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 22:43:21.267045 24821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:21.267141 24821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:21.307257 24821 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43397
I20260807 22:43:21.307353 24959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43397 every 8 connection(s)
I20260807 22:43:21.308688 24821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:21.310241 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 24821
I20260807 22:43:21.310427 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:21.311050 20522 external_mini_cluster.cc:1428] Setting key f42840b825d75b0f7a23f66f54e7350e
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:21.334323 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.312071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:40345 (local address 127.20.10.190:38009)
0807 22:43:21.312852 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.312864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.314259 (+ 1395us) server_negotiation.cc:407] Connection header received
0807 22:43:21.316670 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.316676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.316759 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.316886 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.318919 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.319874 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.321829 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.322256 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.322356 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.326980 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.326986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.326991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.327043 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.330452 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.331438 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.331442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.331445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.331578 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.332331 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.332334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.332336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.332671 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.332802 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.333739 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.333941 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":480,"threads_started":1}
I20260807 22:43:21.335361 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.312857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:38009 (local address 127.20.10.131:40345)
0807 22:43:21.313959 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:21.314027 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:21.316350 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:21.317284 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:21.317295 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:21.317946 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:21.318715 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.318753 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.320071 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.320075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.321653 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.321669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.322503 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.322509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.322869 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.323705 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:21.323732 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:21.326636 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:21.330641 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:21.330711 (+ 70us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:21.330759 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:21.331298 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:21.331945 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:21.331949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:21.331952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:21.332199 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:21.332834 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:21.332841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:21.333321 (+ 480us) client_negotiation.cc:769] Sending connection context
0807 22:43:21.333568 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 22:43:21.334009 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":496,"threads_started":1}
I20260807 22:43:21.337010 24960 heartbeater.cc:347] Connected to a master server at 127.20.10.190:38009
I20260807 22:43:21.337327 24960 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:21.338083 24960 heartbeater.cc:511] Master 127.20.10.190:38009 requested a full tablet report, sending...
I20260807 22:43:21.340119 24458 ts_manager.cc:194] Registered new tserver with Master: 9dc329d90eec47cb81120617b8458233 (127.20.10.131:43397)
I20260807 22:43:21.341230 24458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:40345
I20260807 22:43:21.341897 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:21.406126 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.387981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.20.10.129:34055)
0807 22:43:21.388750 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.388758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.388787 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:43:21.388855 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.388862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.389103 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.389258 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.391045 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.392347 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.393790 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.394540 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.394779 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.399639 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.399659 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.399676 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.399720 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.402995 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.403830 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.403838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.403840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.403972 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.404371 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.404376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.404379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.404748 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.405109 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.405455 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.405748 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":421,"threads_started":1}
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:21.429517 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.411894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.20.10.190:38009)
0807 22:43:21.412154 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.412160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.412398 (+ 238us) server_negotiation.cc:407] Connection header received
0807 22:43:21.412514 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.412519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.412647 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.412753 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.414511 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.415357 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.416836 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.417428 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.417548 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.422518 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.422525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.422528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.422573 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.425768 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.426848 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.426856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.426859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.427102 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.427636 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.427642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.427645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.428251 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.428539 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.428892 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.429159 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260807 22:43:21.430236 24458 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33516
I20260807 22:43:21.431643 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24529
I20260807 22:43:21.447067 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.432880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.20.10.129:34055)
0807 22:43:21.433084 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.433088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.433125 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:43:21.433441 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.433446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.433527 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.433613 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.435178 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.435933 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.437329 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.437785 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.437888 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.440669 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.440677 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.440681 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.440724 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.443945 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.444993 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.444997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.444999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.445166 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.445553 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.445557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.445558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.445878 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.446087 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.446503 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.446766 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:43:21.447824 24663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:21.537786 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24529
I20260807 22:43:21.561896 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24676
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:43:21.581790 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.563373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.20.10.130:35083)
0807 22:43:21.564009 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.564016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.564043 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:21.564130 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.564137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.564435 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.564588 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.566135 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.567456 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.568973 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.569563 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.569676 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.574609 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.574637 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.574712 (+ 75us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.574781 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.578516 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.579613 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.579621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.579623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.579780 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.580204 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.580208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.580210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.580650 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.580897 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.581218 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.581484 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":353,"threads_started":1}
I20260807 22:43:21.583256 24810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:21.671047 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24676
I20260807 22:43:21.696620 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24821
Aug 07 22:43:21 dist-test-slave-csg5 krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:43:21.716710 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.697917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.20.10.131:43397)
0807 22:43:21.698970 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.698977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.699009 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:43:21.699073 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.699080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.699394 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.699595 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.701210 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.702325 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.703796 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.704427 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.704584 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.709711 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:21.709731 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:21.709749 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:21.709797 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:21.713298 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.714244 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.714253 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.714256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.714410 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:21.714896 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:21.714902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:21.714907 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:21.715314 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:21.715669 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:21.716017 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:43:21.716340 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":555,"threads_started":1}
I20260807 22:43:21.718197 24955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:21.806934 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24821
I20260807 22:43:21.827980 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 24424
I20260807 22:43:21.829481 24489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:21.941459 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 24424
2026-08-07T22:43:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4517 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:22 dist-test-slave-csg5 krb5kdc[24985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:22 dist-test-slave-csg5 krb5kdc[24985](info): set up 2 sockets
Aug 07 22:43:22 dist-test-slave-csg5 krb5kdc[24985](info): commencing operation
krb5kdc: starting...
W20260807 22:43:23.162247 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:23 dist-test-slave-csg5 krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:23.440896 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34435
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:34435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:23.810431 25000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:23.810851 25000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:23.811022 25000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:23.818915 25000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:23.819051 25000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:23.819134 25000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:23.819370 25000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:23.819489 25000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:23.830447 25000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:34435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:23.831930 25000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:23.833369 25000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:23.846365 25006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:23.846330 25005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:23.846517 25008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:23.847409 25000 server_base.cc:1034] running on GCE node
I20260807 22:43:23.848393 25000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:23.850194 25000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:23.851432 25000 hybrid_clock.cc:648] HybridClock initialized: now 1786142603851391 us; error 37 us; skew 500 ppm
I20260807 22:43:23.851900 25000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:23 dist-test-slave-csg5 krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:23.860359 25000 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:23.861734 25000 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:23.861840 25000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:23.862138 25000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:23.865469 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "557aa5541de14daa90a50e57ca454a2e"
format_stamp: "Formatted at 2026-08-07 22:43:23 on dist-test-slave-csg5"
server_key: "13e0382b87a0fb5353a9d27e141b0239"
server_key_iv: "cb7a12bb042af8d8eeeb2d1ecc0cd030"
server_key_version: "encryptionkey@0"
I20260807 22:43:23.866302 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "557aa5541de14daa90a50e57ca454a2e"
format_stamp: "Formatted at 2026-08-07 22:43:23 on dist-test-slave-csg5"
server_key: "13e0382b87a0fb5353a9d27e141b0239"
server_key_iv: "cb7a12bb042af8d8eeeb2d1ecc0cd030"
server_key_version: "encryptionkey@0"
I20260807 22:43:23.872254 25000 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 22:43:23.876535 25015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:23.878299 25000 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:43:23.878538 25000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "557aa5541de14daa90a50e57ca454a2e"
format_stamp: "Formatted at 2026-08-07 22:43:23 on dist-test-slave-csg5"
server_key: "13e0382b87a0fb5353a9d27e141b0239"
server_key_iv: "cb7a12bb042af8d8eeeb2d1ecc0cd030"
server_key_version: "encryptionkey@0"
I20260807 22:43:23.878873 25000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:23.894369 25000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:23.899847 25000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:23.900228 25000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:23.923274 25000 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:34435
I20260807 22:43:23.923372 25072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:34435 every 8 connection(s)
I20260807 22:43:23.924373 25000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:23.928654 25073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:23.928843 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25000
I20260807 22:43:23.929051 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:23.929890 20522 external_mini_cluster.cc:1428] Setting key 39ca1201ad8ad1797983f8543e312813
I20260807 22:43:23.944726 25073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: Bootstrap starting.
Aug 07 22:43:23 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:23.948632 25073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:23.950268 25073 log.cc:824] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:23.954335 25073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: No bootstrap required, opened a new log
I20260807 22:43:23.958559 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:23.932335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.20.10.190:34435)
0807 22:43:23.933258 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:23.933275 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:23.933327 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:43:23.936187 (+ 2860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:23.936222 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:23.936838 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:23.937339 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:23.939008 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.941109 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.942711 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.943840 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.944033 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.948342 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:23.948357 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:23.948382 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:23.948440 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:23.952836 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:23.953795 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:23.953804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:23.953813 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:23.954051 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:23.954689 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:23.954698 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:23.954704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:23.955640 (+ 936us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:23.955964 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:23.956430 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 22:43:23.956973 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":395,"threads_started":1}
I20260807 22:43:23.964625 25073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } }
I20260807 22:43:23.965045 25073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:23.965148 25073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557aa5541de14daa90a50e57ca454a2e, State: Initialized, Role: FOLLOWER
I20260807 22:43:23.965662 25073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } }
I20260807 22:43:23.965865 25073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:23.965962 25073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:23.966092 25073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:23.968760 25073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } }
I20260807 22:43:23.969316 25073 leader_election.cc:302] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 557aa5541de14daa90a50e57ca454a2e; no voters:
I20260807 22:43:23.969975 25073 leader_election.cc:288] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:23.970230 25079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:23.971108 25079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [term 1 LEADER]: Becoming Leader. State: Replica: 557aa5541de14daa90a50e57ca454a2e, State: Running, Role: LEADER
I20260807 22:43:23.971741 25079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } }
I20260807 22:43:23.972308 25073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:23.974804 25081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 557aa5541de14daa90a50e57ca454a2e. Latest consensus state: current_term: 1 leader_uuid: "557aa5541de14daa90a50e57ca454a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } } }
I20260807 22:43:23.975462 25081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:23.975054 25080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "557aa5541de14daa90a50e57ca454a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557aa5541de14daa90a50e57ca454a2e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34435 } } }
I20260807 22:43:23.976118 25080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:23.977936 25085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:23.985134 25085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:23.995666 25085 catalog_manager.cc:1347] Generated new cluster ID: ded6b159e54a4bb6817e45aa362a9501
I20260807 22:43:23.995829 25085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:24.018680 25085 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:24.020226 25085 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:24.036729 25085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: Generated new TSK 0
I20260807 22:43:24.037751 25085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:43:24.108570 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:24.516717 25101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:24.517136 25101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:24.517350 25101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:24.526132 25101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:24.526362 25101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:24.526616 25101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:24.539073 25101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:24.540637 25101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:24.542191 25101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:24.558804 25107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:24.558845 25110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:24.558732 25108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:24.559846 25101 server_base.cc:1034] running on GCE node
I20260807 22:43:24.560539 25101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:24.561689 25101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:24.563054 25101 hybrid_clock.cc:648] HybridClock initialized: now 1786142604562953 us; error 77 us; skew 500 ppm
I20260807 22:43:24.563525 25101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:24 dist-test-slave-csg5 krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:24.567588 25101 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:43:24.569226 25101 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:24.569350 25101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:24.569819 25101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:24.574051 25101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "8621b88e04f048109e9cb72295c25f84"
format_stamp: "Formatted at 2026-08-07 22:43:24 on dist-test-slave-csg5"
server_key: "82a107694f7ce900046bf2862f9ab7f6"
server_key_iv: "6ce209e21d3110e9dcf157945ffb5e67"
server_key_version: "encryptionkey@0"
I20260807 22:43:24.575167 25101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "8621b88e04f048109e9cb72295c25f84"
format_stamp: "Formatted at 2026-08-07 22:43:24 on dist-test-slave-csg5"
server_key: "82a107694f7ce900046bf2862f9ab7f6"
server_key_iv: "6ce209e21d3110e9dcf157945ffb5e67"
server_key_version: "encryptionkey@0"
I20260807 22:43:24.583143 25101 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260807 22:43:24.588461 25118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:24.590372 25101 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 22:43:24.590751 25101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "8621b88e04f048109e9cb72295c25f84"
format_stamp: "Formatted at 2026-08-07 22:43:24 on dist-test-slave-csg5"
server_key: "82a107694f7ce900046bf2862f9ab7f6"
server_key_iv: "6ce209e21d3110e9dcf157945ffb5e67"
server_key_version: "encryptionkey@0"
I20260807 22:43:24.591064 25101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:24.626318 25101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:24.633077 25101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:24.633481 25101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:24.634785 25101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:24.636577 25101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:24.636726 25101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:24.636884 25101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:24.636987 25101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:24.679872 25101 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41621
I20260807 22:43:24.679941 25239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41621 every 8 connection(s)
I20260807 22:43:24.681175 25101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:24.685005 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25101
I20260807 22:43:24.685262 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:24.686195 20522 external_mini_cluster.cc:1428] Setting key a88b2d436556c32a2e41d8ac05b09ddc
Aug 07 22:43:24 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142604, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:24.712895 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:24.684568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:58523 (local address 127.20.10.190:34435)
0807 22:43:24.685304 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:24.685314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:24.686950 (+ 1636us) server_negotiation.cc:407] Connection header received
0807 22:43:24.689834 (+ 2884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:24.689843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:24.690055 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:24.690329 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:24.693126 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.695003 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.698872 (+ 3869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.699334 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.699508 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.704619 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:24.704625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:24.704628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:24.704670 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:24.708446 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:24.709744 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:24.709750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:24.709752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:24.709918 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:24.710505 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:24.710510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:24.710512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:24.710953 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:24.711211 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:24.712154 (+ 943us) server_negotiation.cc:299] Negotiation successful
0807 22:43:24.712459 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":386,"threads_started":1}
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:43:24.714010 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:24.685446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34435 (local address 127.20.10.129:58523)
0807 22:43:24.686734 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:24.686804 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:24.689395 (+ 2591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:24.690608 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:24.690625 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:24.691791 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:24.692852 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.692887 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.696059 (+ 3172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.696068 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.698550 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.698572 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.699662 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.699672 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.700031 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.700993 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:24.701041 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:24.704273 (+ 3232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:24.708787 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:24.708796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:24.708814 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:24.709531 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:24.710072 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:24.710080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:24.710089 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:24.710337 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:24.711121 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:24.711132 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:24.711650 (+ 518us) client_negotiation.cc:769] Sending connection context
0807 22:43:24.711991 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 22:43:24.712685 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":580,"threads_started":1}
I20260807 22:43:24.716568 25240 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34435
I20260807 22:43:24.717154 25240 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:43:24.718050 25240 heartbeater.cc:511] Master 127.20.10.190:34435 requested a full tablet report, sending...
I20260807 22:43:24.721006 25033 ts_manager.cc:194] Registered new tserver with Master: 8621b88e04f048109e9cb72295c25f84 (127.20.10.129:41621)
I20260807 22:43:24.722601 25033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:58523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:43:24.753382 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:25.139282 25246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:25.139595 25246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:25.139809 25246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:25.147740 25246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:25.147887 25246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:25.148084 25246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:25.158906 25246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:25.160343 25246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:25.161741 25246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:25.177973 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:25.178368 25252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:25.178517 25255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:25.179531 25246 server_base.cc:1034] running on GCE node
I20260807 22:43:25.180410 25246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:25.181532 25246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:25.182801 25246 hybrid_clock.cc:648] HybridClock initialized: now 1786142605182699 us; error 73 us; skew 500 ppm
I20260807 22:43:25.183225 25246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:25 dist-test-slave-csg5 krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142605, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:25.187104 25246 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:25.188387 25246 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:25.188494 25246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:25.188782 25246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:25.192638 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "ab2bf79c2700cc697126a95faf5633f5"
server_key_iv: "cb8a87fb4cc537a35ab435aada40ff74"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.193866 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "ab2bf79c2700cc697126a95faf5633f5"
server_key_iv: "cb8a87fb4cc537a35ab435aada40ff74"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.200941 25246 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 22:43:25.205467 25262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.207557 25246 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:43:25.207765 25246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "ab2bf79c2700cc697126a95faf5633f5"
server_key_iv: "cb8a87fb4cc537a35ab435aada40ff74"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.208019 25246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:25.225821 25246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:25.231137 25246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:25.231482 25246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:25.232612 25246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:25.234201 25246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:25.234297 25246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.234422 25246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:25.234495 25246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.274325 25246 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:38133
I20260807 22:43:25.274379 25386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:38133 every 8 connection(s)
I20260807 22:43:25.275396 25246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:25.282989 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25246
I20260807 22:43:25.283197 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:25.283826 20522 external_mini_cluster.cc:1428] Setting key 8101ddb60d2ae6435b0c8375857c19df
Aug 07 22:43:25 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142605, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:25.303247 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.278513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:55011 (local address 127.20.10.190:34435)
0807 22:43:25.279037 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.279046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.280175 (+ 1129us) server_negotiation.cc:407] Connection header received
0807 22:43:25.282351 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.282359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.282490 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.282604 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.284979 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.285785 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.288633 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.289105 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.289229 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.294750 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:25.294760 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:25.294768 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:25.294826 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:25.298230 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.299540 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.299546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.299548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.299767 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.300493 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.300498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.300502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.300938 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:25.301101 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.302503 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.302839 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":248,"threads_started":1}
I20260807 22:43:25.304872 25390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.279127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34435 (local address 127.20.10.130:55011)
0807 22:43:25.279992 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.280035 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.281966 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.282968 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.282983 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.283872 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:25.284745 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.284773 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.285970 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.285974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.288340 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.288368 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.289400 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.289410 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.289790 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.290723 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:25.290761 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:25.294305 (+ 3544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:25.298453 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:25.298466 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:25.298491 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:25.299325 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:25.299926 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:25.299934 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:25.299941 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:25.300245 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:25.301113 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:25.301125 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:25.301933 (+ 808us) client_negotiation.cc:769] Sending connection context
0807 22:43:25.302364 (+ 431us) client_negotiation.cc:240] Negotiation successful
0807 22:43:25.303147 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":403,"threads_started":1}
I20260807 22:43:25.307125 25387 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34435
I20260807 22:43:25.307691 25387 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:25.308826 25387 heartbeater.cc:511] Master 127.20.10.190:34435 requested a full tablet report, sending...
I20260807 22:43:25.310747 25033 ts_manager.cc:194] Registered new tserver with Master: 65d70222cdda4fa2b1c6e6c7d2ace47a (127.20.10.130:38133)
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:43:25.311789 25033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:55011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:25.350755 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:25.726145 25240 heartbeater.cc:503] Master 127.20.10.190:34435 was elected leader, sending a full tablet report...
W20260807 22:43:25.732705 25393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:25.733089 25393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:25.733291 25393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:25.740950 25393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:25.741154 25393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:25.741370 25393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:25.752146 25393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:25.753806 25393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:25.755414 25393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:25.770493 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:25.770527 25399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:25.770447 25401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:25.771835 25393 server_base.cc:1034] running on GCE node
I20260807 22:43:25.772710 25393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:25.773988 25393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:25.775251 25393 hybrid_clock.cc:648] HybridClock initialized: now 1786142605775170 us; error 76 us; skew 500 ppm
I20260807 22:43:25.775699 25393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:25 dist-test-slave-csg5 krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142605, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:25.779613 25393 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:25.780944 25393 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:25.781052 25393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:25.781355 25393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:25.784866 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "77671e4dd09f618e420bc6000b090558"
server_key_iv: "131cccc4601d893a8bde375200b32498"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.785889 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "77671e4dd09f618e420bc6000b090558"
server_key_iv: "131cccc4601d893a8bde375200b32498"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.792080 25393 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:43:25.796819 25407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.798768 25393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:43:25.798986 25393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7"
format_stamp: "Formatted at 2026-08-07 22:43:25 on dist-test-slave-csg5"
server_key: "77671e4dd09f618e420bc6000b090558"
server_key_iv: "131cccc4601d893a8bde375200b32498"
server_key_version: "encryptionkey@0"
I20260807 22:43:25.799285 25393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:25.813791 25393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:25.818835 25393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:25.819172 25393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:25.820344 25393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:25.821921 25393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:25.822012 25393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.822181 25393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:25.822265 25393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:25.862020 25393 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:41651
I20260807 22:43:25.862082 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:41651 every 8 connection(s)
I20260807 22:43:25.863095 25393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:25.868394 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25393
I20260807 22:43:25.868609 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:25.869091 20522 external_mini_cluster.cc:1428] Setting key 5d4d3467fab54ba46821ec2a21232f72
Aug 07 22:43:25 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142605, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:25.887133 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.865950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:60023 (local address 127.20.10.190:34435)
0807 22:43:25.866476 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.866481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.867565 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 22:43:25.869793 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.869800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.869921 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.870060 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.871888 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.872869 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.874633 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.875128 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.875290 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.879773 (+ 4483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:25.879779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:25.879783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:25.879824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:25.883169 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.884273 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.884279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.884288 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.884430 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.884959 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.884963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.884966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.885330 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:25.885526 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.886358 (+ 832us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.886630 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":240,"threads_started":1}
I20260807 22:43:25.888257 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.866532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34435 (local address 127.20.10.131:60023)
0807 22:43:25.867375 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.867418 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.869451 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.870346 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.870356 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.871039 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:25.871666 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.871686 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.873054 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.873058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.874465 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.874479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.875398 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.875408 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.875658 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.876508 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:25.876538 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:25.879458 (+ 2920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:25.883394 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:25.883412 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:25.883430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:25.884112 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:25.884564 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:25.884571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:25.884575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:25.884813 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:25.885492 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:25.885502 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:25.885934 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 22:43:25.886244 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 22:43:25.886891 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":321,"threads_started":1}
I20260807 22:43:25.890470 25529 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34435
I20260807 22:43:25.890878 25529 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:25.891642 25529 heartbeater.cc:511] Master 127.20.10.190:34435 requested a full tablet report, sending...
I20260807 22:43:25.893476 25033 ts_manager.cc:194] Registered new tserver with Master: 70d17ff3ac3b46d8af09bcbcc37f41e7 (127.20.10.131:41651)
I20260807 22:43:25.894457 25033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:60023
I20260807 22:43:25.898517 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:25.919133 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.904319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.20.10.190:34435)
0807 22:43:25.904556 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.904561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.905254 (+ 693us) server_negotiation.cc:407] Connection header received
0807 22:43:25.905351 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.905355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.905441 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.905522 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.907089 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.907924 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.909397 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.909877 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.909986 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.912834 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:25.912840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:25.912844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:25.912886 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:25.915954 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.916759 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.916764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.916766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.916916 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:25.917471 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:25.917476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:25.917478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:25.917886 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:25.918156 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.918522 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.918843 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:43:25.924708 25033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:25.933612 25033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:25.987618 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.948629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:41651 (local address 127.0.0.1:55048)
0807 22:43:25.974517 (+ 25888us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.974611 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.974955 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.978464 (+ 3509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.978474 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.978511 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:25.979915 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.979932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.981670 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.981675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.984343 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.984365 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.986394 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.986400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.986617 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.986719 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 22:43:25.986882 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 22:43:25.987110 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25479,"mutex_wait_us":44,"thread_start_us":29243,"threads_started":1}
I20260807 22:43:25.987979 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.949456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38133 (local address 127.0.0.1:57336)
0807 22:43:25.978190 (+ 28734us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.978218 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.978371 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.979136 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.979143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.979174 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:25.980590 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.980601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.982751 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.982760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.985029 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.985043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.986957 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.986963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.987146 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.987176 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 22:43:25.987281 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:43:25.987415 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28577,"mutex_wait_us":63,"thread_start_us":28051,"threads_started":1}
I20260807 22:43:25.988839 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.949002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.20.10.131:41651)
0807 22:43:25.977610 (+ 28608us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.977618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.977653 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:25.977771 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.977784 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.978169 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.978810 (+ 641us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:25.980242 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.981500 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.984578 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.986180 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.986349 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.986631 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.988187 (+ 1556us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.988398 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28437,"thread_start_us":28245,"threads_started":1}
I20260807 22:43:25.989307 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.949066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57752)
0807 22:43:25.977408 (+ 28342us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.977439 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.977818 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.979185 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.979191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.979227 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:25.980347 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.980366 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.982603 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.982608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.985758 (+ 3150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.985773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.988294 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.988302 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.988536 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.988584 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:43:25.988677 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 22:43:25.988829 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28117,"mutex_wait_us":46,"thread_start_us":26274,"threads_started":1}
I20260807 22:43:25.989369 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.949924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.20.10.129:41621)
0807 22:43:25.975433 (+ 25509us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.975448 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.977613 (+ 2165us) server_negotiation.cc:407] Connection header received
0807 22:43:25.978204 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.978233 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.978491 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.978731 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:25.980549 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.982440 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.985967 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.987682 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.987859 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.988104 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.988818 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.989020 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23258,"thread_start_us":23516,"threads_started":1}
I20260807 22:43:25.990190 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.950159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.20.10.130:38133)
0807 22:43:25.976377 (+ 26218us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.976392 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.978368 (+ 1976us) server_negotiation.cc:407] Connection header received
0807 22:43:25.978531 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.978542 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.978817 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.979039 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:25.980817 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.982580 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.985257 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.986707 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.986872 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.987080 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:25.987367 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 22:43:25.987580 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25922,"thread_start_us":26240,"threads_started":1}
I20260807 22:43:25.993196 25461 tablet_service.cc:1467] Processing CreateTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee (DEFAULT_TABLE table=test-table [id=d8755aca8f754f729e051fed675ab5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:25.993851 25315 tablet_service.cc:1467] Processing CreateTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee (DEFAULT_TABLE table=test-table [id=d8755aca8f754f729e051fed675ab5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:25.993851 25171 tablet_service.cc:1467] Processing CreateTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee (DEFAULT_TABLE table=test-table [id=d8755aca8f754f729e051fed675ab5c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:25.994748 25461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 448fba41f5ac4137aa42222bc6aa1eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:25.995360 25171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 448fba41f5ac4137aa42222bc6aa1eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:25.995385 25315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 448fba41f5ac4137aa42222bc6aa1eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:26.008988 25550 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Bootstrap starting.
I20260807 22:43:26.012969 25552 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Bootstrap starting.
I20260807 22:43:26.013230 25550 tablet_bootstrap.cc:654] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:26.017045 25550 log.cc:824] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:26.018952 25551 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Bootstrap starting.
I20260807 22:43:26.019199 25552 tablet_bootstrap.cc:654] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:26.020954 25552 log.cc:824] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:26.022166 25550 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: No bootstrap required, opened a new log
I20260807 22:43:26.022516 25550 ts_tablet_manager.cc:1397] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260807 22:43:26.023809 25551 tablet_bootstrap.cc:654] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:26.025128 25552 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: No bootstrap required, opened a new log
I20260807 22:43:26.025344 25551 log.cc:824] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:26.025581 25552 ts_tablet_manager.cc:1397] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260807 22:43:26.029476 25551 tablet_bootstrap.cc:492] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: No bootstrap required, opened a new log
I20260807 22:43:26.029862 25551 ts_tablet_manager.cc:1397] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 22:43:26.034278 25550 raft_consensus.cc:348] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.034879 25550 raft_consensus.cc:374] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:26.035041 25550 raft_consensus.cc:729] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d17ff3ac3b46d8af09bcbcc37f41e7, State: Initialized, Role: FOLLOWER
I20260807 22:43:26.035051 25552 raft_consensus.cc:348] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.035624 25552 raft_consensus.cc:374] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:26.035799 25552 raft_consensus.cc:729] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d70222cdda4fa2b1c6e6c7d2ace47a, State: Initialized, Role: FOLLOWER
I20260807 22:43:26.036003 25550 consensus_queue.cc:260] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.036540 25552 consensus_queue.cc:260] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.038470 25387 heartbeater.cc:503] Master 127.20.10.190:34435 was elected leader, sending a full tablet report...
I20260807 22:43:26.039216 25529 heartbeater.cc:503] Master 127.20.10.190:34435 was elected leader, sending a full tablet report...
I20260807 22:43:26.040045 25550 ts_tablet_manager.cc:1428] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260807 22:43:26.040642 25552 ts_tablet_manager.cc:1428] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Time spent starting tablet: real 0.015s user 0.007s sys 0.005s
I20260807 22:43:26.044425 25551 raft_consensus.cc:348] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.044935 25551 raft_consensus.cc:374] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:26.045069 25551 raft_consensus.cc:729] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8621b88e04f048109e9cb72295c25f84, State: Initialized, Role: FOLLOWER
I20260807 22:43:26.046069 25551 consensus_queue.cc:260] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.048517 25551 ts_tablet_manager.cc:1428] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
W20260807 22:43:26.114945 25530 tablet.cc:2369] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:26.184446 25241 tablet.cc:2369] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:26.201581 25556 raft_consensus.cc:482] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:26.201952 25556 raft_consensus.cc:504] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.203997 25556 leader_election.cc:288] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8621b88e04f048109e9cb72295c25f84 (127.20.10.129:41621), 65d70222cdda4fa2b1c6e6c7d2ace47a (127.20.10.130:38133)
I20260807 22:43:26.211719 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.204743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:53641 (local address 127.20.10.130:38133)
0807 22:43:26.205057 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.205065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.205108 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:43:26.205201 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.205206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.205290 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.205400 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:26.206054 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.207112 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.209195 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.210521 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.210730 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.210809 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.211278 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.211427 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260807 22:43:26.211781 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.204382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38133 (local address 127.20.10.131:53641)
0807 22:43:26.204780 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.204804 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:26.204990 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:26.205501 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:26.205510 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:26.205562 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:26.205873 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.205890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.207300 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.207306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.208979 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.208993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.210794 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.210802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.211023 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.211062 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:43:26.211177 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:43:26.211340 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"spinlock_wait_cycles":1280}
I20260807 22:43:26.212985 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.204383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:56659 (local address 127.20.10.129:41621)
0807 22:43:26.204630 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.204636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.205549 (+ 913us) server_negotiation.cc:407] Connection header received
0807 22:43:26.205714 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.205720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.205810 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.205926 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:26.206778 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.207805 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.210490 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.211920 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.212153 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.212250 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.212576 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.212717 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:43:26.213078 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.204454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.20.10.131:56659)
0807 22:43:26.205394 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.205412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:26.205544 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:26.205950 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:26.205980 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:26.206009 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:26.206463 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.206479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.208055 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.208059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.210231 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.210248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.212229 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.212235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.212396 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.212420 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:43:26.212541 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:43:26.212693 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":506,"threads_started":1}
I20260807 22:43:26.213208 25336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "448fba41f5ac4137aa42222bc6aa1eee" candidate_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" is_pre_election: true
I20260807 22:43:26.213974 25336 raft_consensus.cc:2454] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70d17ff3ac3b46d8af09bcbcc37f41e7 in term 0.
I20260807 22:43:26.214250 25193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "448fba41f5ac4137aa42222bc6aa1eee" candidate_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8621b88e04f048109e9cb72295c25f84" is_pre_election: true
I20260807 22:43:26.214968 25193 raft_consensus.cc:2454] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70d17ff3ac3b46d8af09bcbcc37f41e7 in term 0.
I20260807 22:43:26.215001 25408 leader_election.cc:302] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65d70222cdda4fa2b1c6e6c7d2ace47a, 70d17ff3ac3b46d8af09bcbcc37f41e7; no voters:
I20260807 22:43:26.215595 25556 raft_consensus.cc:2781] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:26.215834 25556 raft_consensus.cc:482] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:26.216001 25556 raft_consensus.cc:3037] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:26.218636 25556 raft_consensus.cc:504] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.219535 25556 leader_election.cc:288] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [CANDIDATE]: Term 1 election: Requested vote from peers 8621b88e04f048109e9cb72295c25f84 (127.20.10.129:41621), 65d70222cdda4fa2b1c6e6c7d2ace47a (127.20.10.130:38133)
I20260807 22:43:26.220336 25336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "448fba41f5ac4137aa42222bc6aa1eee" candidate_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a"
I20260807 22:43:26.220433 25193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "448fba41f5ac4137aa42222bc6aa1eee" candidate_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8621b88e04f048109e9cb72295c25f84"
I20260807 22:43:26.220681 25193 raft_consensus.cc:3037] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:26.220649 25336 raft_consensus.cc:3037] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:26.224748 25193 raft_consensus.cc:2454] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70d17ff3ac3b46d8af09bcbcc37f41e7 in term 1.
I20260807 22:43:26.224756 25336 raft_consensus.cc:2454] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70d17ff3ac3b46d8af09bcbcc37f41e7 in term 1.
I20260807 22:43:26.225579 25411 leader_election.cc:302] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 65d70222cdda4fa2b1c6e6c7d2ace47a, 70d17ff3ac3b46d8af09bcbcc37f41e7, 8621b88e04f048109e9cb72295c25f84; no voters:
I20260807 22:43:26.226608 25556 raft_consensus.cc:2781] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:26.227645 25556 raft_consensus.cc:686] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 1 LEADER]: Becoming Leader. State: Replica: 70d17ff3ac3b46d8af09bcbcc37f41e7, State: Running, Role: LEADER
I20260807 22:43:26.228672 25556 consensus_queue.cc:237] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } }
I20260807 22:43:26.234897 25032 catalog_manager.cc:5495] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70d17ff3ac3b46d8af09bcbcc37f41e7 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70d17ff3ac3b46d8af09bcbcc37f41e7" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 } health_report { overall_health: UNKNOWN } } }
W20260807 22:43:26.277906 25388 tablet.cc:2369] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:26.301498 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.294467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.20.10.131:41651)
0807 22:43:26.294843 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.294850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.295006 (+ 156us) server_negotiation.cc:407] Connection header received
0807 22:43:26.295403 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.295412 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.295534 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.295743 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:26.296203 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.297064 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.298689 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.299419 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.299623 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.300300 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:26.300843 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:26.301041 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.301110 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.301205 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260807 22:43:26.310830 25336 raft_consensus.cc:1261] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 1 FOLLOWER]: Refusing update from remote peer 70d17ff3ac3b46d8af09bcbcc37f41e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:26.310830 25193 raft_consensus.cc:1261] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 1 FOLLOWER]: Refusing update from remote peer 70d17ff3ac3b46d8af09bcbcc37f41e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:26.312364 25560 consensus_queue.cc:1035] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8621b88e04f048109e9cb72295c25f84" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:26.312896 25556 consensus_queue.cc:1035] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d70222cdda4fa2b1c6e6c7d2ace47a" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:26.327962 25564 mvcc.cc:205] Tried to move back new op lower bound from 7316040115436949504 to 7316040115118088192. Current Snapshot: MvccSnapshot[applied={T|T < 7316040115436949504}]
I20260807 22:43:26.350240 25032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:26.350700 25032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:26.357586 25032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 557aa5541de14daa90a50e57ca454a2e: Sending DeleteTablet for 3 replicas of tablet 448fba41f5ac4137aa42222bc6aa1eee
I20260807 22:43:26.359517 25171 tablet_service.cc:1514] Processing DeleteTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:26 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260807 22:43:26.360129 25315 tablet_service.cc:1514] Processing DeleteTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:26 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:57336
I20260807 22:43:26.360536 25461 tablet_service.cc:1514] Processing DeleteTablet for tablet 448fba41f5ac4137aa42222bc6aa1eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:26 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:55048
I20260807 22:43:26.362495 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25101
I20260807 22:43:26.372671 25575 tablet_replica.cc:330] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: stopping tablet replica
I20260807 22:43:26.376415 25575 raft_consensus.cc:2229] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:26.377116 25575 raft_consensus.cc:2258] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:26.378702 25576 tablet_replica.cc:330] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: stopping tablet replica
I20260807 22:43:26.379242 25576 raft_consensus.cc:2229] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:26.379560 25575 ts_tablet_manager.cc:1905] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:26.380050 25576 raft_consensus.cc:2258] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:26.380999 25574 tablet_replica.cc:330] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: stopping tablet replica
I20260807 22:43:26.381594 25574 raft_consensus.cc:2229] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:26.382190 25574 raft_consensus.cc:2258] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:26.385366 25576 ts_tablet_manager.cc:1905] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:43:26 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:26.388458 25575 ts_tablet_manager.cc:1918] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:26.388695 25575 log.cc:1196] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/448fba41f5ac4137aa42222bc6aa1eee
I20260807 22:43:26.389753 25575 ts_tablet_manager.cc:1939] T 448fba41f5ac4137aa42222bc6aa1eee P 65d70222cdda4fa2b1c6e6c7d2ace47a: Deleting consensus metadata
I20260807 22:43:26.392479 25016 catalog_manager.cc:4841] TS 65d70222cdda4fa2b1c6e6c7d2ace47a (127.20.10.130:38133): tablet 448fba41f5ac4137aa42222bc6aa1eee (table test-table [id=d8755aca8f754f729e051fed675ab5c4]) successfully deleted
I20260807 22:43:26.394254 25574 ts_tablet_manager.cc:1905] T 448fba41f5ac4137aa42222bc6aa1eee P 8621b88e04f048109e9cb72295c25f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:26.396670 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.364512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.20.10.129:41621)
0807 22:43:26.364799 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.364808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.372010 (+ 7202us) server_negotiation.cc:407] Connection header received
0807 22:43:26.373584 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.373593 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.373756 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.373893 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.376143 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.377738 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.380444 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.381204 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.381321 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.388465 (+ 7144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:26.388491 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:26.388511 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:26.388569 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:26.393068 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.393937 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.393948 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.393951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.394149 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.394710 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.394716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.394720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.395161 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:26.395533 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.395905 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.396193 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 22:43:26.397526 25235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:26.402024 25576 ts_tablet_manager.cc:1918] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:26.402294 25576 log.cc:1196] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/448fba41f5ac4137aa42222bc6aa1eee
I20260807 22:43:26.403076 25576 ts_tablet_manager.cc:1939] T 448fba41f5ac4137aa42222bc6aa1eee P 70d17ff3ac3b46d8af09bcbcc37f41e7: Deleting consensus metadata
I20260807 22:43:26.405411 25017 catalog_manager.cc:4841] TS 70d17ff3ac3b46d8af09bcbcc37f41e7 (127.20.10.131:41651): tablet 448fba41f5ac4137aa42222bc6aa1eee (table test-table [id=d8755aca8f754f729e051fed675ab5c4]) successfully deleted
I20260807 22:43:26.495050 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25101
I20260807 22:43:26.528613 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25246
W20260807 22:43:26.528604 25019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.10.129:41621 (error 108)
I20260807 22:43:26.534003 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.532584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57772)
0807 22:43:26.533604 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.533777 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":737,"threads_started":1}
W20260807 22:43:26.534521 25019 catalog_manager.cc:4584] TS 8621b88e04f048109e9cb72295c25f84 (127.20.10.129:41621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 448fba41f5ac4137aa42222bc6aa1eee: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Aug 07 22:43:26 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:43:26.550926 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.529788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.20.10.130:38133)
0807 22:43:26.530011 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.530019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.530403 (+ 384us) server_negotiation.cc:407] Connection header received
0807 22:43:26.530496 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.530503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.530613 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.530795 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.532314 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.533746 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.536843 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.537541 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.537675 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.543543 (+ 5868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:26.543563 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:26.543579 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:26.543623 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:26.547103 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.548109 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.548118 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.548120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.548373 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.548847 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.548850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.548853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.549334 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:26.549579 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.550032 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.550349 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:43:26.551815 25382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:26.588840 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.588136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57784)
0807 22:43:26.588490 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.588619 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260807 22:43:26.590327 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.589676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57788)
0807 22:43:26.590028 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.590124 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260807 22:43:26.648098 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25246
I20260807 22:43:26.653450 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.652843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57796)
0807 22:43:26.653118 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.653236 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:43:26.654896 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.654301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57804)
0807 22:43:26.654506 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.654629 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 22:43:26.673533 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25393
Aug 07 22:43:26 dist-test-slave-csg5 krb5kdc[24985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:43:26.695158 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.674977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.20.10.131:41651)
0807 22:43:26.675206 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.675214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.675239 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:26.675328 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.675335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.675399 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.675528 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.676950 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.678023 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.679616 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.680116 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.680281 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.685845 (+ 5564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:26.685866 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:26.685882 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:26.685936 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:26.690743 (+ 4807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.691977 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.692007 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.692009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.692202 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:26.692745 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:26.692756 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:26.692758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:26.693192 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:26.693543 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:26.693955 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 22:43:26.694245 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:43:26.696879 25524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:26.736116 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.735507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57812)
0807 22:43:26.735766 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.735896 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:43:26.737609 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.737062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57826)
0807 22:43:26.737320 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.737420 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 22:43:26.835914 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25393
I20260807 22:43:26.862602 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25000
I20260807 22:43:26.864028 25065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:26.985208 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.984476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41621 (local address 127.0.0.1:57828)
0807 22:43:26.984769 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.984897 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 22:43:26.985248 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25000
2026-08-07T22:43:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5043 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T22:43:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:27Z Disabled control of system clock
I20260807 22:43:27.074359 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35697
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:27.453465 25587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:27.453814 25587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:27.453907 25587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:27.461755 25587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:27.461891 25587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:27.461933 25587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:27.461987 25587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:27.462033 25587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:27.472293 25587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:27.473690 25587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:27.476042 25587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:27.492564 25592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:27.492486 25596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:27.492425 25593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:27.494451 25587 server_base.cc:1034] running on GCE node
I20260807 22:43:27.495652 25587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:27.497732 25587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:27.499054 25587 hybrid_clock.cc:648] HybridClock initialized: now 1786142607498999 us; error 69 us; skew 500 ppm
I20260807 22:43:27.499617 25587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:27.502975 25587 webserver.cc:460] Webserver started at http://127.20.10.190:43613/ using document root <none> and password file <none>
I20260807 22:43:27.503839 25587 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:27.503998 25587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:27.504477 25587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:27.508201 25587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "e8f39b65549b45b19b304ad9e657684c"
format_stamp: "Formatted at 2026-08-07 22:43:27 on dist-test-slave-csg5"
server_key: "5a747374cfc773a7857a4c6fa0d19b1c"
server_key_iv: "029b9910fba5c2dd6bc52a1415cf144c"
server_key_version: "encryptionkey@0"
I20260807 22:43:27.509202 25587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "e8f39b65549b45b19b304ad9e657684c"
format_stamp: "Formatted at 2026-08-07 22:43:27 on dist-test-slave-csg5"
server_key: "5a747374cfc773a7857a4c6fa0d19b1c"
server_key_iv: "029b9910fba5c2dd6bc52a1415cf144c"
server_key_version: "encryptionkey@0"
I20260807 22:43:27.515771 25587 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:43:27.520269 25602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:27.522015 25587 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:43:27.522277 25587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "e8f39b65549b45b19b304ad9e657684c"
format_stamp: "Formatted at 2026-08-07 22:43:27 on dist-test-slave-csg5"
server_key: "5a747374cfc773a7857a4c6fa0d19b1c"
server_key_iv: "029b9910fba5c2dd6bc52a1415cf144c"
server_key_version: "encryptionkey@0"
I20260807 22:43:27.522625 25587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:27.529475 25587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:27.530478 25587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:27.530872 25587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:27.553144 25587 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:35697
I20260807 22:43:27.553160 25662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:35697 every 8 connection(s)
I20260807 22:43:27.554922 25587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:27.559746 25663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:27.563871 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25587
I20260807 22:43:27.564173 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:27.564970 20522 external_mini_cluster.cc:1428] Setting key 705e595ee5ed598daf5066458afbb136
I20260807 22:43:27.575333 25663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: Bootstrap starting.
I20260807 22:43:27.577932 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:27.567592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42892 (local address 127.20.10.190:35697)
0807 22:43:27.568906 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:27.568924 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:27.568987 (+ 63us) server_negotiation.cc:407] Connection header received
0807 22:43:27.572580 (+ 3593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:27.572628 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:27.573395 (+ 767us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:27.573981 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:27.575501 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:27.575511 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:27.575531 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:27.575558 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:27.575656 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:27.575683 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:27.575966 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:27.576121 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:43:27.576463 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":439,"threads_started":1}
I20260807 22:43:27.580368 25663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:27.581859 25663 log.cc:824] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:27.585464 25663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: No bootstrap required, opened a new log
I20260807 22:43:27.593840 25663 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } }
I20260807 22:43:27.594406 25663 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:27.594573 25663 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f39b65549b45b19b304ad9e657684c, State: Initialized, Role: FOLLOWER
I20260807 22:43:27.595340 25663 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } }
I20260807 22:43:27.595633 25663 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:27.595798 25663 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:27.595995 25663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:27.598464 25663 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } }
I20260807 22:43:27.599126 25663 leader_election.cc:302] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8f39b65549b45b19b304ad9e657684c; no voters:
I20260807 22:43:27.600178 25663 leader_election.cc:288] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:27.600330 25669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:27.601277 25669 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [term 1 LEADER]: Becoming Leader. State: Replica: e8f39b65549b45b19b304ad9e657684c, State: Running, Role: LEADER
I20260807 22:43:27.602001 25669 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } }
I20260807 22:43:27.602943 25663 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:27.605876 25670 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8f39b65549b45b19b304ad9e657684c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } } }
I20260807 22:43:27.606419 25670 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:27.606227 25672 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8f39b65549b45b19b304ad9e657684c. Latest consensus state: current_term: 1 leader_uuid: "e8f39b65549b45b19b304ad9e657684c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f39b65549b45b19b304ad9e657684c" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35697 } } }
I20260807 22:43:27.606606 25672 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:27.609018 25677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:27.615398 25677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:27.625481 25677 catalog_manager.cc:1347] Generated new cluster ID: b9f95aacd9bc45be85dca8438157a6f0
I20260807 22:43:27.625716 25677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:27.643456 25677 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:27.644018 25677 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:27.651283 25677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: Generated new TSK 0
I20260807 22:43:27.652349 25677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:27.660343 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--builtin_ntp_servers=127.20.10.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:28.056186 25689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:28.056573 25689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:28.056720 25689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:28.065229 25689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:28.065384 25689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:28.065624 25689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:28.076295 25689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:28.077796 25689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:28.079865 25689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:28.096511 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:28.096616 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:28.096567 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:28.097716 25689 server_base.cc:1034] running on GCE node
I20260807 22:43:28.098802 25689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:28.100018 25689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:28.101271 25689 hybrid_clock.cc:648] HybridClock initialized: now 1786142608101228 us; error 42 us; skew 500 ppm
I20260807 22:43:28.101768 25689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:28.104895 25689 webserver.cc:460] Webserver started at http://127.20.10.129:33079/ using document root <none> and password file <none>
I20260807 22:43:28.105746 25689 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:28.105854 25689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:28.106227 25689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:28.110019 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "c85ab73cea7644d9b996b56d6e7c0b86"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "9aaf027838e80e8dd762e00038503dcd"
server_key_iv: "ad4e5a8553723163516a24071af0c4fa"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.111105 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "c85ab73cea7644d9b996b56d6e7c0b86"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "9aaf027838e80e8dd762e00038503dcd"
server_key_iv: "ad4e5a8553723163516a24071af0c4fa"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.117537 25689 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:28.121824 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.123404 25689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:28.123611 25689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "c85ab73cea7644d9b996b56d6e7c0b86"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "9aaf027838e80e8dd762e00038503dcd"
server_key_iv: "ad4e5a8553723163516a24071af0c4fa"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.123939 25689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:28.130900 25689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:28.131923 25689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:28.132257 25689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:28.133368 25689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:28.135223 25689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:28.135360 25689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.135504 25689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:28.135612 25689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.175627 25689 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:37057
I20260807 22:43:28.175714 25827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:37057 every 8 connection(s)
I20260807 22:43:28.177793 25689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:28.179044 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25689
I20260807 22:43:28.179312 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:28.180191 20522 external_mini_cluster.cc:1428] Setting key b085285212c224a7fd48ca2a127a17e7
I20260807 22:43:28.184024 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--builtin_ntp_servers=127.20.10.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 22:43:28.190644 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.181526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:50089 (local address 127.20.10.190:35697)
0807 22:43:28.182076 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:28.182083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:28.183990 (+ 1907us) server_negotiation.cc:407] Connection header received
0807 22:43:28.187503 (+ 3513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:28.187508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:28.187580 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:28.187717 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:28.189623 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:28.189628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:28.189632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:28.189662 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:28.189714 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:28.189731 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:28.189873 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:28.190185 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 22:43:28.190359 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":273,"threads_started":1}
I20260807 22:43:28.192041 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.182179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35697 (local address 127.20.10.129:50089)
0807 22:43:28.183771 (+ 1592us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:28.183826 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:28.187028 (+ 3202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:28.188065 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:28.188081 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:28.188236 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:28.189127 (+ 891us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:28.189133 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:28.189204 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:28.189368 (+ 164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:28.189863 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:28.189877 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:28.189888 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:43:28.190212 (+ 324us) client_negotiation.cc:240] Negotiation successful
0807 22:43:28.190551 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":570,"threads_started":1}
I20260807 22:43:28.194087 25828 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35697
I20260807 22:43:28.194455 25828 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:28.195291 25828 heartbeater.cc:511] Master 127.20.10.190:35697 requested a full tablet report, sending...
I20260807 22:43:28.198167 25622 ts_manager.cc:194] Registered new tserver with Master: c85ab73cea7644d9b996b56d6e7c0b86 (127.20.10.129:37057)
W20260807 22:43:28.579797 25833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:28.580174 25833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:28.580376 25833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:28.588415 25833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:28.588594 25833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:28.588800 25833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:28.598744 25833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:28.600602 25833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:28.603559 25833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:28.619863 25841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:28.619992 25840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:28.619907 25843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:28.621606 25833 server_base.cc:1034] running on GCE node
I20260807 22:43:28.622462 25833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:28.623816 25833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:28.625113 25833 hybrid_clock.cc:648] HybridClock initialized: now 1786142608624966 us; error 130 us; skew 500 ppm
I20260807 22:43:28.625605 25833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:28.628696 25833 webserver.cc:460] Webserver started at http://127.20.10.130:44979/ using document root <none> and password file <none>
I20260807 22:43:28.629531 25833 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:28.629690 25833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:28.630092 25833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:28.633390 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "23f1c835b5104b337a09352f20b84865"
server_key_iv: "d07c2e9ca2de1e77df5ba0bfc9622a7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.634368 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "23f1c835b5104b337a09352f20b84865"
server_key_iv: "d07c2e9ca2de1e77df5ba0bfc9622a7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.640363 25833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:28.644649 25849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.646319 25833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:43:28.646538 25833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915"
format_stamp: "Formatted at 2026-08-07 22:43:28 on dist-test-slave-csg5"
server_key: "23f1c835b5104b337a09352f20b84865"
server_key_iv: "d07c2e9ca2de1e77df5ba0bfc9622a7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:28.646857 25833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:28.653196 25833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:28.654130 25833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:28.654492 25833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:28.655630 25833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:28.657331 25833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:28.657470 25833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.657626 25833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:28.657724 25833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:28.697276 25833 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44339
I20260807 22:43:28.697389 25970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44339 every 8 connection(s)
I20260807 22:43:28.698889 25833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:28.706043 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25833
I20260807 22:43:28.706243 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:28.706820 20522 external_mini_cluster.cc:1428] Setting key 09dbe21f9f3a611950231f050a92624f
I20260807 22:43:28.710289 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.702223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:48127 (local address 127.20.10.190:35697)
0807 22:43:28.702888 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:28.702893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:28.704642 (+ 1749us) server_negotiation.cc:407] Connection header received
0807 22:43:28.707095 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:28.707101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:28.707193 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:28.707341 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:28.709217 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:28.709223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:28.709228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:28.709248 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:28.709294 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:28.709308 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:28.709460 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:28.709870 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 22:43:28.709946 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":262,"threads_started":1}
I20260807 22:43:28.710759 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--builtin_ntp_servers=127.20.10.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 22:43:28.711560 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.703066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35697 (local address 127.20.10.130:48127)
0807 22:43:28.704425 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:28.704495 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:28.706586 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:28.707702 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:28.707716 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:28.707890 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:28.708799 (+ 909us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:28.708802 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:28.708848 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:28.708956 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:28.709482 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:28.709498 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:28.709509 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:43:28.709918 (+ 409us) client_negotiation.cc:240] Negotiation successful
0807 22:43:28.710182 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":343,"threads_started":1}
I20260807 22:43:28.713305 25971 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35697
I20260807 22:43:28.713676 25971 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:28.714497 25971 heartbeater.cc:511] Master 127.20.10.190:35697 requested a full tablet report, sending...
I20260807 22:43:28.716499 25622 ts_manager.cc:194] Registered new tserver with Master: 1e51dbedd74f47c1aa9a2f97a2bc1915 (127.20.10.130:44339)
W20260807 22:43:29.080545 25975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:29.080873 25975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:29.081028 25975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:29.088745 25975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:29.088872 25975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:29.089078 25975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:29.099097 25975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:29.100569 25975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:29.102777 25975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:29.119405 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:29.119457 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:29.119500 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:29.121002 25975 server_base.cc:1034] running on GCE node
I20260807 22:43:29.121731 25975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:29.123010 25975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:29.124327 25975 hybrid_clock.cc:648] HybridClock initialized: now 1786142609124268 us; error 53 us; skew 500 ppm
I20260807 22:43:29.124703 25975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:29.131222 25975 webserver.cc:460] Webserver started at http://127.20.10.131:42865/ using document root <none> and password file <none>
I20260807 22:43:29.132069 25975 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:29.132232 25975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:29.132613 25975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:29.136008 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "4b29757983c64de79d0ca73969b3609f"
format_stamp: "Formatted at 2026-08-07 22:43:29 on dist-test-slave-csg5"
server_key: "cdbba0a7eaf4b8dc0f3ef560558a1b77"
server_key_iv: "7f0a1544d5fcfaa7e92da90b8915e2d0"
server_key_version: "encryptionkey@0"
I20260807 22:43:29.136904 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "4b29757983c64de79d0ca73969b3609f"
format_stamp: "Formatted at 2026-08-07 22:43:29 on dist-test-slave-csg5"
server_key: "cdbba0a7eaf4b8dc0f3ef560558a1b77"
server_key_iv: "7f0a1544d5fcfaa7e92da90b8915e2d0"
server_key_version: "encryptionkey@0"
I20260807 22:43:29.142798 25975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:43:29.146875 25990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:29.148480 25975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:29.148693 25975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "4b29757983c64de79d0ca73969b3609f"
format_stamp: "Formatted at 2026-08-07 22:43:29 on dist-test-slave-csg5"
server_key: "cdbba0a7eaf4b8dc0f3ef560558a1b77"
server_key_iv: "7f0a1544d5fcfaa7e92da90b8915e2d0"
server_key_version: "encryptionkey@0"
I20260807 22:43:29.148964 25975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:29.155885 25975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:29.156824 25975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:29.157150 25975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:29.158197 25975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:29.159860 25975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:29.160007 25975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:29.160166 25975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:29.160269 25975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:29.201989 25828 heartbeater.cc:503] Master 127.20.10.190:35697 was elected leader, sending a full tablet report...
I20260807 22:43:29.205072 25975 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:36577
I20260807 22:43:29.205241 26116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:36577 every 8 connection(s)
I20260807 22:43:29.206543 25975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:29.210229 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 25975
I20260807 22:43:29.210444 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:29.211163 20522 external_mini_cluster.cc:1428] Setting key e7918a8dc0de92f62514df4a7fa0315d
I20260807 22:43:29.216625 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.209662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:33793 (local address 127.20.10.190:35697)
0807 22:43:29.209884 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.209889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.211994 (+ 2105us) server_negotiation.cc:407] Connection header received
0807 22:43:29.214055 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.214060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.214133 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.214235 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.215829 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.215835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.215840 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.215861 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.215907 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.215919 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.216057 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.216279 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.216365 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:43:29.217700 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.210454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35697 (local address 127.20.10.131:33793)
0807 22:43:29.211770 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.211832 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.213713 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.214505 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.214514 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.214616 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.215499 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.215502 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.215544 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.215640 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.216034 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.216045 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.216053 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.216368 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.216670 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":479,"threads_started":1}
I20260807 22:43:29.219751 26117 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35697
I20260807 22:43:29.220083 26117 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:29.220722 26117 heartbeater.cc:511] Master 127.20.10.190:35697 requested a full tablet report, sending...
I20260807 22:43:29.222388 25622 ts_manager.cc:194] Registered new tserver with Master: 4b29757983c64de79d0ca73969b3609f (127.20.10.131:36577)
I20260807 22:43:29.228497 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:29.237634 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.234064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.20.10.190:35697)
0807 22:43:29.234354 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.234362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.234958 (+ 596us) server_negotiation.cc:407] Connection header received
0807 22:43:29.235288 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.235294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.235468 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.235635 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.236845 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.236849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.236852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.236871 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.236914 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.236926 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.237074 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.237267 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.237346 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:43:29.242447 25622 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:29.249917 25622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:29.282632 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.265947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.20.10.131:36577)
0807 22:43:29.267936 (+ 1989us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.267950 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.277988 (+ 10038us) server_negotiation.cc:407] Connection header received
0807 22:43:29.278270 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.278280 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.278481 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.278603 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.281483 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.281498 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.281520 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.281548 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.281639 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.281671 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.281877 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.282006 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.282142 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1716,"thread_start_us":1470,"threads_started":1}
I20260807 22:43:29.282774 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.265288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50078)
0807 22:43:29.268994 (+ 3706us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.269048 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.270789 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.279583 (+ 8794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.279593 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.279700 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.281169 (+ 1469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.281173 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.281187 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.281232 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.281945 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.281951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.281956 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.282121 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.282190 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3429,"thread_start_us":3238,"threads_started":1}
I20260807 22:43:29.283648 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.265588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.20.10.129:37057)
0807 22:43:29.277468 (+ 11880us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.277481 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.277525 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:43:29.277616 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.277628 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.277961 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.278151 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.281523 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.281535 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.281549 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.281576 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.281667 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.281693 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.281882 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.283049 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.283181 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11647,"thread_start_us":11472,"threads_started":1}
I20260807 22:43:29.283738 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.265490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:36577 (local address 127.0.0.1:50108)
0807 22:43:29.277824 (+ 12334us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.277846 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.278102 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.279764 (+ 1662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.279770 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.279811 (+ 41us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.281062 (+ 1251us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.281067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.281118 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.281232 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.281783 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.281799 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.281810 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.282030 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.282143 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12145,"thread_start_us":15871,"threads_started":1}
I20260807 22:43:29.284971 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.271184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:44339 (local address 127.0.0.1:54388)
0807 22:43:29.281430 (+ 10246us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.281458 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.281603 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.282337 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.282344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.282379 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.283585 (+ 1206us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.283588 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.283610 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.283663 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.284292 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.284297 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.284303 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.284443 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.284501 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10063,"thread_start_us":6300,"threads_started":1}
I20260807 22:43:29.285742 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.269896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.20.10.130:44339)
0807 22:43:29.280956 (+ 11060us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.280969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.281685 (+ 716us) server_negotiation.cc:407] Connection header received
0807 22:43:29.281794 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.281804 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.282108 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.282265 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.283874 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.283884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.283899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.283932 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.283996 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.284023 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.284217 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.285307 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.285425 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10844,"thread_start_us":10689,"threads_started":1}
I20260807 22:43:29.288322 25758 tablet_service.cc:1467] Processing CreateTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 (DEFAULT_TABLE table=test-table [id=9819955b031e4eff8f7a275d4da7d918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:29.288420 26045 tablet_service.cc:1467] Processing CreateTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 (DEFAULT_TABLE table=test-table [id=9819955b031e4eff8f7a275d4da7d918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:29.289788 25758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bf42a7661f4603900b3ac0ff6bd835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:29.290045 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bf42a7661f4603900b3ac0ff6bd835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:29.289729 25899 tablet_service.cc:1467] Processing CreateTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 (DEFAULT_TABLE table=test-table [id=9819955b031e4eff8f7a275d4da7d918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:29.291328 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bf42a7661f4603900b3ac0ff6bd835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:29.304363 26136 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Bootstrap starting.
I20260807 22:43:29.307262 26138 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Bootstrap starting.
I20260807 22:43:29.309232 26139 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Bootstrap starting.
I20260807 22:43:29.308394 26136 tablet_bootstrap.cc:654] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:29.311403 26138 tablet_bootstrap.cc:654] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:29.311568 26136 log.cc:824] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:29.312853 26138 log.cc:824] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:29.312955 26139 tablet_bootstrap.cc:654] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:29.314298 26139 log.cc:824] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:29.315299 26136 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: No bootstrap required, opened a new log
I20260807 22:43:29.315629 26136 ts_tablet_manager.cc:1397] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260807 22:43:29.316174 26138 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: No bootstrap required, opened a new log
I20260807 22:43:29.316459 26138 ts_tablet_manager.cc:1397] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 22:43:29.317078 26139 tablet_bootstrap.cc:492] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: No bootstrap required, opened a new log
I20260807 22:43:29.317349 26139 ts_tablet_manager.cc:1397] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260807 22:43:29.325852 26139 raft_consensus.cc:348] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.326330 26139 raft_consensus.cc:374] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:29.326483 26139 raft_consensus.cc:729] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e51dbedd74f47c1aa9a2f97a2bc1915, State: Initialized, Role: FOLLOWER
I20260807 22:43:29.327310 26139 consensus_queue.cc:260] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.328055 26138 raft_consensus.cc:348] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.328055 26136 raft_consensus.cc:348] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.328562 26136 raft_consensus.cc:374] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:29.328552 26138 raft_consensus.cc:374] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:29.328681 26136 raft_consensus.cc:729] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85ab73cea7644d9b996b56d6e7c0b86, State: Initialized, Role: FOLLOWER
I20260807 22:43:29.328672 26138 raft_consensus.cc:729] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b29757983c64de79d0ca73969b3609f, State: Initialized, Role: FOLLOWER
I20260807 22:43:29.329423 26138 consensus_queue.cc:260] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.329423 26136 consensus_queue.cc:260] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.331895 26117 heartbeater.cc:503] Master 127.20.10.190:35697 was elected leader, sending a full tablet report...
I20260807 22:43:29.332705 26138 ts_tablet_manager.cc:1428] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260807 22:43:29.334090 25971 heartbeater.cc:503] Master 127.20.10.190:35697 was elected leader, sending a full tablet report...
I20260807 22:43:29.334764 26139 ts_tablet_manager.cc:1428] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260807 22:43:29.337347 26136 ts_tablet_manager.cc:1428] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Time spent starting tablet: real 0.022s user 0.016s sys 0.003s
W20260807 22:43:29.430994 25830 tablet.cc:2369] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:29.431155 26143 raft_consensus.cc:482] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:29.431643 26143 raft_consensus.cc:504] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.433581 26143 leader_election.cc:288] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c85ab73cea7644d9b996b56d6e7c0b86 (127.20.10.129:37057), 4b29757983c64de79d0ca73969b3609f (127.20.10.131:36577)
I20260807 22:43:29.438041 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.434288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:38505 (local address 127.20.10.131:36577)
0807 22:43:29.434493 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.434499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.435576 (+ 1077us) server_negotiation.cc:407] Connection header received
0807 22:43:29.435786 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.435790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.435850 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.435977 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.437205 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.437213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.437221 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.437245 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.437307 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.437324 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.437440 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.437587 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.437661 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:43:29.438104 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.434065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44943 (local address 127.20.10.129:37057)
0807 22:43:29.434311 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.434318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.435524 (+ 1206us) server_negotiation.cc:407] Connection header received
0807 22:43:29.435735 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.435741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.435808 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.435894 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.437352 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.437370 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.437377 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.437436 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.437481 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.437503 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.437632 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.437756 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.437824 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 22:43:29.438304 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.434262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.20.10.130:44943)
0807 22:43:29.435387 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.435408 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.435570 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.435970 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.435974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.435998 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.437092 (+ 1094us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.437095 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.437131 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.437180 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.437616 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.437621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.437626 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.437780 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.437832 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":554,"threads_started":1}
I20260807 22:43:29.438764 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.434260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:36577 (local address 127.20.10.130:38505)
0807 22:43:29.435417 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.435432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.435624 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.436034 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.436039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.436066 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.436993 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:29.436996 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.437010 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:29.437054 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:29.437423 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:29.437428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:29.437433 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:43:29.437599 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 22:43:29.437663 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":31,"spinlock_wait_cycles":51968,"thread_start_us":620,"threads_started":1}
I20260807 22:43:29.439585 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bf42a7661f4603900b3ac0ff6bd835" candidate_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" is_pre_election: true
I20260807 22:43:29.439762 26066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bf42a7661f4603900b3ac0ff6bd835" candidate_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b29757983c64de79d0ca73969b3609f" is_pre_election: true
I20260807 22:43:29.440204 25778 raft_consensus.cc:2454] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e51dbedd74f47c1aa9a2f97a2bc1915 in term 0.
I20260807 22:43:29.440337 26066 raft_consensus.cc:2454] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e51dbedd74f47c1aa9a2f97a2bc1915 in term 0.
I20260807 22:43:29.441092 25850 leader_election.cc:302] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1e51dbedd74f47c1aa9a2f97a2bc1915, 4b29757983c64de79d0ca73969b3609f, c85ab73cea7644d9b996b56d6e7c0b86; no voters:
I20260807 22:43:29.441571 26143 raft_consensus.cc:2781] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:29.441725 26143 raft_consensus.cc:482] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:29.441828 26143 raft_consensus.cc:3037] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:29.444363 26143 raft_consensus.cc:504] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
I20260807 22:43:29.445132 26143 leader_election.cc:288] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [CANDIDATE]: Term 1 election: Requested vote from peers c85ab73cea7644d9b996b56d6e7c0b86 (127.20.10.129:37057), 4b29757983c64de79d0ca73969b3609f (127.20.10.131:36577)
I20260807 22:43:29.445775 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bf42a7661f4603900b3ac0ff6bd835" candidate_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85ab73cea7644d9b996b56d6e7c0b86"
I20260807 22:43:29.445794 26066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bf42a7661f4603900b3ac0ff6bd835" candidate_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b29757983c64de79d0ca73969b3609f"
I20260807 22:43:29.446028 25778 raft_consensus.cc:3037] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:29.446064 26066 raft_consensus.cc:3037] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:29.448642 25778 raft_consensus.cc:2454] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e51dbedd74f47c1aa9a2f97a2bc1915 in term 1.
I20260807 22:43:29.448891 26066 raft_consensus.cc:2454] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e51dbedd74f47c1aa9a2f97a2bc1915 in term 1.
I20260807 22:43:29.449256 25850 leader_election.cc:302] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e51dbedd74f47c1aa9a2f97a2bc1915, c85ab73cea7644d9b996b56d6e7c0b86; no voters:
I20260807 22:43:29.449682 26143 raft_consensus.cc:2781] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:29.450237 26143 raft_consensus.cc:686] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 1 LEADER]: Becoming Leader. State: Replica: 1e51dbedd74f47c1aa9a2f97a2bc1915, State: Running, Role: LEADER
I20260807 22:43:29.450892 26143 consensus_queue.cc:237] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } }
W20260807 22:43:29.451448 25972 tablet.cc:2369] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:29.457305 25620 catalog_manager.cc:5495] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e51dbedd74f47c1aa9a2f97a2bc1915 (127.20.10.130). New cstate: current_term: 1 leader_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e51dbedd74f47c1aa9a2f97a2bc1915" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44339 } health_report { overall_health: HEALTHY } } }
W20260807 22:43:29.458524 26118 tablet.cc:2369] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:29.477975 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.474992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.20.10.130:44339)
0807 22:43:29.475288 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.475293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.475318 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:29.475474 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.475479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.475571 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.475679 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.476822 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.476827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.476830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.476853 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.476888 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.476901 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.477013 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.477466 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.477551 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 22:43:29.488751 26066 raft_consensus.cc:1261] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 1 FOLLOWER]: Refusing update from remote peer 1e51dbedd74f47c1aa9a2f97a2bc1915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:29.488754 25778 raft_consensus.cc:1261] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 1 FOLLOWER]: Refusing update from remote peer 1e51dbedd74f47c1aa9a2f97a2bc1915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:29.489938 26148 consensus_queue.cc:1035] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b29757983c64de79d0ca73969b3609f" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:29.490379 26143 consensus_queue.cc:1035] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c85ab73cea7644d9b996b56d6e7c0b86" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:29.526089 25621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:29.526463 25621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:29.532433 25621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8f39b65549b45b19b304ad9e657684c: Sending DeleteTablet for 3 replicas of tablet a6bf42a7661f4603900b3ac0ff6bd835
I20260807 22:43:29.534013 25758 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:29 UTC) from {username='slave'} at 127.0.0.1:50078
I20260807 22:43:29.534197 25899 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:29 UTC) from {username='slave'} at 127.0.0.1:54388
I20260807 22:43:29.534960 26045 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bf42a7661f4603900b3ac0ff6bd835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:29 UTC) from {username='slave'} at 127.0.0.1:50108
I20260807 22:43:29.535689 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25689
I20260807 22:43:29.545630 26162 tablet_replica.cc:330] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: stopping tablet replica
I20260807 22:43:29.547112 26162 raft_consensus.cc:2229] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:29.548534 26162 raft_consensus.cc:2258] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:29.549305 26163 tablet_replica.cc:330] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: stopping tablet replica
I20260807 22:43:29.550284 26163 raft_consensus.cc:2229] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:29.552632 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.537957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.20.10.129:37057)
0807 22:43:29.538236 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.538245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.548381 (+ 10136us) server_negotiation.cc:407] Connection header received
0807 22:43:29.549037 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.549048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.549196 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.549295 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.551361 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.551371 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.551379 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.551412 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.551470 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.551493 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.551651 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.552047 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.552141 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 22:43:29.553458 26163 raft_consensus.cc:2258] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:29.554751 26164 tablet_replica.cc:330] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: stopping tablet replica
I20260807 22:43:29.554920 25823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:29.555286 26164 raft_consensus.cc:2229] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:29.555958 26163 ts_tablet_manager.cc:1905] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:29.556269 26164 raft_consensus.cc:2258] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:29.559973 26164 ts_tablet_manager.cc:1905] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:29.564769 26163 ts_tablet_manager.cc:1918] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:29.565449 26163 log.cc:1196] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/a6bf42a7661f4603900b3ac0ff6bd835
I20260807 22:43:29.566162 26163 ts_tablet_manager.cc:1939] T a6bf42a7661f4603900b3ac0ff6bd835 P 1e51dbedd74f47c1aa9a2f97a2bc1915: Deleting consensus metadata
I20260807 22:43:29.568328 25604 catalog_manager.cc:4841] TS 1e51dbedd74f47c1aa9a2f97a2bc1915 (127.20.10.130:44339): tablet a6bf42a7661f4603900b3ac0ff6bd835 (table test-table [id=9819955b031e4eff8f7a275d4da7d918]) successfully deleted
I20260807 22:43:29.570158 26164 ts_tablet_manager.cc:1918] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:29.570371 26164 log.cc:1196] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/a6bf42a7661f4603900b3ac0ff6bd835
I20260807 22:43:29.571225 26164 ts_tablet_manager.cc:1939] T a6bf42a7661f4603900b3ac0ff6bd835 P 4b29757983c64de79d0ca73969b3609f: Deleting consensus metadata
I20260807 22:43:29.573745 25605 catalog_manager.cc:4841] TS 4b29757983c64de79d0ca73969b3609f (127.20.10.131:36577): tablet a6bf42a7661f4603900b3ac0ff6bd835 (table test-table [id=9819955b031e4eff8f7a275d4da7d918]) successfully deleted
I20260807 22:43:29.563030 26162 ts_tablet_manager.cc:1905] T a6bf42a7661f4603900b3ac0ff6bd835 P c85ab73cea7644d9b996b56d6e7c0b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:29.668035 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25689
W20260807 22:43:29.694388 25604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.10.129:37057 (error 108)
I20260807 22:43:29.694459 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25833
I20260807 22:43:29.698278 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.697556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50092)
0807 22:43:29.697911 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.698078 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
W20260807 22:43:29.698607 25604 catalog_manager.cc:4584] TS c85ab73cea7644d9b996b56d6e7c0b86 (127.20.10.129:37057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6bf42a7661f4603900b3ac0ff6bd835: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
I20260807 22:43:29.699820 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.695538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.20.10.130:44339)
0807 22:43:29.695773 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.695781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.696226 (+ 445us) server_negotiation.cc:407] Connection header received
0807 22:43:29.696345 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.696352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.696426 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.696531 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.697922 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.697929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.697935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.697956 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.697998 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.698014 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.698148 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.699108 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.699188 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:43:29.700984 25966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:29.753275 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.752612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50102)
0807 22:43:29.752951 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.753070 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 22:43:29.754552 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.754134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50118)
0807 22:43:29.754300 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.754388 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 22:43:29.815327 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.814752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50130)
0807 22:43:29.815028 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.815158 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 22:43:29.816468 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.816068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50136)
0807 22:43:29.816241 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.816335 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 22:43:29.828871 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25833
I20260807 22:43:29.851092 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25975
I20260807 22:43:29.854924 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.852096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.20.10.131:36577)
0807 22:43:29.852481 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.852488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.852509 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:43:29.852614 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.852620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.852768 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.852874 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.854088 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:29.854094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:29.854099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:29.854118 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:29.854160 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:29.854173 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:29.854455 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:29.854513 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:43:29.854581 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260807 22:43:29.855850 26112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:29.897719 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.896950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50144)
0807 22:43:29.897235 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.897500 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 22:43:29.899240 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.898626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50150)
0807 22:43:29.898963 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.899071 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260807 22:43:29.952646 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25975
I20260807 22:43:29.975178 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 25587
I20260807 22:43:29.976540 25658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:30.095180 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.094620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37057 (local address 127.0.0.1:50152)
0807 22:43:30.094885 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:30.095000 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 22:43:30.095189 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 25587
2026-08-07T22:43:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3109 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 22:43:30.147740 20522 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:30.151187 20522 webserver.cc:460] Webserver started at https://127.0.0.1:33819/ using document root <none> and password file <none>
I20260807 22:43:30.151665 20522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:30.152460 20522 webserver.cc:460] Webserver started at http://127.0.0.1:32987/ using document root <none> and password file <none>
2026-08-07T22:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:30Z Disabled control of system clock
I20260807 22:43:30.195343 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34971
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:42777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:34971
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33819/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/testchainca.cert with env {}
W20260807 22:43:30.569697 26176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:30.570053 26176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:30.570228 26176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:30.578258 26176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:30.578403 26176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:30.578444 26176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:30.578562 26176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:30.578689 26176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:30.578811 26176 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:30.578876 26176 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33819/jwks.json
I20260807 22:43:30.590409 26176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:34971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33819/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:30.592507 26176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:30.594050 26176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:30.609642 26185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:30.609653 26182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:30.610193 26183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:30.611851 26176 server_base.cc:1034] running on GCE node
I20260807 22:43:30.613157 26176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:30.615042 26176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:30.616374 26176 hybrid_clock.cc:648] HybridClock initialized: now 1786142610616322 us; error 46 us; skew 500 ppm
I20260807 22:43:30.616851 26176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:30.620477 26176 webserver.cc:460] Webserver started at http://127.20.10.190:45351/ using document root <none> and password file <none>
I20260807 22:43:30.621358 26176 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:30.621479 26176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:30.621783 26176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:30.625340 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "64ae9a2aa5224a968b10dd464634cf07"
format_stamp: "Formatted at 2026-08-07 22:43:30 on dist-test-slave-csg5"
server_key: "d0506b5581cab66e71adf71a6828d702"
server_key_iv: "913c4a17eaa94a1e6605d7b45b820134"
server_key_version: "encryptionkey@0"
I20260807 22:43:30.626194 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "64ae9a2aa5224a968b10dd464634cf07"
format_stamp: "Formatted at 2026-08-07 22:43:30 on dist-test-slave-csg5"
server_key: "d0506b5581cab66e71adf71a6828d702"
server_key_iv: "913c4a17eaa94a1e6605d7b45b820134"
server_key_version: "encryptionkey@0"
I20260807 22:43:30.632349 26176 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260807 22:43:30.636423 26192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:30.638087 26176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:43:30.638334 26176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "64ae9a2aa5224a968b10dd464634cf07"
format_stamp: "Formatted at 2026-08-07 22:43:30 on dist-test-slave-csg5"
server_key: "d0506b5581cab66e71adf71a6828d702"
server_key_iv: "913c4a17eaa94a1e6605d7b45b820134"
server_key_version: "encryptionkey@0"
I20260807 22:43:30.638622 26176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:30.656643 26176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:30.657732 26176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:30.658135 26176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:30.681802 26176 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:34971
I20260807 22:43:30.681836 26245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:34971 every 8 connection(s)
I20260807 22:43:30.683486 26176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:30.687866 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:30.694111 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26176
I20260807 22:43:30.694380 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:30.695159 20522 external_mini_cluster.cc:1428] Setting key fa7a417fabe09c445b87dd304202fd28
I20260807 22:43:30.701700 26246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07: Bootstrap starting.
I20260807 22:43:30.706238 26246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:30.707877 26246 log.cc:824] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:30.712301 26246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07: No bootstrap required, opened a new log
I20260807 22:43:30.713301 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.697730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.20.10.190:34971)
0807 22:43:30.699449 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:30.699464 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:30.699521 (+ 57us) server_negotiation.cc:407] Connection header received
0807 22:43:30.701590 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:30.701616 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:30.702220 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:30.702624 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:30.703485 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.704975 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.707329 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.708119 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.708380 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.710168 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:30.710176 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:30.710193 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:30.710945 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:30.711041 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:30.711065 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:30.711167 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:30.711591 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 22:43:30.712066 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":730,"threads_started":1}
I20260807 22:43:30.717784 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:30.723147 26246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } }
I20260807 22:43:30.723606 26246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:30.723703 26246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ae9a2aa5224a968b10dd464634cf07, State: Initialized, Role: FOLLOWER
I20260807 22:43:30.724314 26246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } }
I20260807 22:43:30.724627 26246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:30.724843 26246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:30.725123 26246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:30.727855 26246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } }
I20260807 22:43:30.728499 26246 leader_election.cc:302] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64ae9a2aa5224a968b10dd464634cf07; no voters:
I20260807 22:43:30.729676 26246 leader_election.cc:288] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:30.729780 26251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:30.730913 26251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [term 1 LEADER]: Becoming Leader. State: Replica: 64ae9a2aa5224a968b10dd464634cf07, State: Running, Role: LEADER
I20260807 22:43:30.731599 26251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } }
I20260807 22:43:30.732681 26246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:30.734794 26253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64ae9a2aa5224a968b10dd464634cf07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } } }
I20260807 22:43:30.735363 26253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:30.734781 26252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ae9a2aa5224a968b10dd464634cf07. Latest consensus state: current_term: 1 leader_uuid: "64ae9a2aa5224a968b10dd464634cf07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ae9a2aa5224a968b10dd464634cf07" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34971 } } }
I20260807 22:43:30.736900 26252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:30.739308 26260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:30.745524 26260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:30.757184 26260 catalog_manager.cc:1347] Generated new cluster ID: ca54cc3c1e1d4a488f5bcd33b72b986c
I20260807 22:43:30.757330 26260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:30.801718 26260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:30.803156 26260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:30.818408 26260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64ae9a2aa5224a968b10dd464634cf07: Generated new TSK 0
I20260807 22:43:30.819721 26260 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:43:30.874768 26280 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 22:43:30.875098 26280 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:43:30.876541 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.848880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.20.10.190:34971)
0807 22:43:30.849162 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:30.849170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:30.849745 (+ 575us) server_negotiation.cc:407] Connection header received
0807 22:43:30.850132 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:30.850138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:30.850240 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:30.850402 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:30.851119 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.852362 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.854160 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.855228 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.855440 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.856219 (+ 779us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:30.875618 (+ 19399us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:43:30.876135 (+ 517us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49314: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 22:43:30.876703 26249 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49314: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 22:43:30.878126 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:30.848770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34971 (local address 127.0.0.1:49314)
0807 22:43:30.849619 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:30.849638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:30.849823 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:30.850449 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:30.850453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:30.850464 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:30.850933 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.850944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.852554 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.852572 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.853985 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.854001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.855503 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.855510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.855741 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.855848 (+ 107us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:30.875994 (+ 20146us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:43:30.876437 (+ 443us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.10.190:34971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":496,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.10.190:34971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33819/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 22:43:30.880640 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26176
I20260807 22:43:30.882377 26241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:30.992650 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26176
2026-08-07T22:43:31Z chronyd exiting
I20260807 22:43:31.051695 20522 test_util.cc:175] -----------------------------------------------
I20260807 22:43:31.051834 20522 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142560216906-20522-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (909 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 22:43:31.057927 20522 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:31.060113 20522 webserver.cc:460] Webserver started at https://127.0.0.1:38335/ using document root <none> and password file <none>
I20260807 22:43:31.060377 20522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:31.061260 20522 webserver.cc:460] Webserver started at http://127.0.0.1:40161/ using document root <none> and password file <none>
2026-08-07T22:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:31Z Disabled control of system clock
I20260807 22:43:31.103329 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:38589
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:39425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:38589
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38335/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/testchainca.cert with env {}
W20260807 22:43:31.504513 26288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:31.504905 26288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:31.505077 26288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:31.513489 26288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:31.513666 26288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:31.513746 26288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:31.513854 26288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:31.513932 26288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:31.514024 26288 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:31.514098 26288 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38335/jwks.json
I20260807 22:43:31.525208 26288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:38589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:38589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38335/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:31.526753 26288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:31.528219 26288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:31.542027 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:31.542068 26294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:31.542161 26296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:31.542841 26288 server_base.cc:1034] running on GCE node
I20260807 22:43:31.543723 26288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:31.545444 26288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:31.546816 26288 hybrid_clock.cc:648] HybridClock initialized: now 1786142611546752 us; error 53 us; skew 500 ppm
I20260807 22:43:31.547250 26288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:31.550117 26288 webserver.cc:460] Webserver started at http://127.20.10.190:42641/ using document root <none> and password file <none>
I20260807 22:43:31.551056 26288 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:31.551208 26288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:31.551566 26288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:31.554898 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "7e254d6d983441f9baf9ac19db0cea0e"
format_stamp: "Formatted at 2026-08-07 22:43:31 on dist-test-slave-csg5"
server_key: "c147a33d92ff702a236e70c92ce62c32"
server_key_iv: "035f3dd0fb29cbeb7902a519368d9cec"
server_key_version: "encryptionkey@0"
I20260807 22:43:31.555774 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "7e254d6d983441f9baf9ac19db0cea0e"
format_stamp: "Formatted at 2026-08-07 22:43:31 on dist-test-slave-csg5"
server_key: "c147a33d92ff702a236e70c92ce62c32"
server_key_iv: "035f3dd0fb29cbeb7902a519368d9cec"
server_key_version: "encryptionkey@0"
I20260807 22:43:31.561681 26288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:43:31.566229 26304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:31.567988 26288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:31.568212 26288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "7e254d6d983441f9baf9ac19db0cea0e"
format_stamp: "Formatted at 2026-08-07 22:43:31 on dist-test-slave-csg5"
server_key: "c147a33d92ff702a236e70c92ce62c32"
server_key_iv: "035f3dd0fb29cbeb7902a519368d9cec"
server_key_version: "encryptionkey@0"
I20260807 22:43:31.568568 26288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:31.589615 26288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:31.590570 26288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:31.590936 26288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:31.613155 26288 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:38589
I20260807 22:43:31.613173 26360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:38589 every 8 connection(s)
I20260807 22:43:31.614802 26288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:31.616814 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26288
I20260807 22:43:31.617115 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:31.618084 20522 external_mini_cluster.cc:1428] Setting key eb6d8917b8d55a0009445ae306cc0618
I20260807 22:43:31.620579 26361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:31.635336 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:31.620407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.20.10.190:38589)
0807 22:43:31.621585 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:31.621597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:31.621642 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:43:31.623401 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:31.623427 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:31.623943 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:31.624346 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:31.625073 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.626583 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.628957 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.629887 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.630062 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.631700 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:31.631709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:31.631725 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:31.632715 (+ 990us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:31.632848 (+ 133us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:31.632878 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:31.633024 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:31.633820 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 22:43:31.634154 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":505,"threads_started":1}
I20260807 22:43:31.636626 26361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e: Bootstrap starting.
I20260807 22:43:31.639853 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:31.640733 26361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:31.642025 26361 log.cc:824] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:31.646009 26361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e: No bootstrap required, opened a new log
I20260807 22:43:31.654587 26361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } }
I20260807 22:43:31.655097 26361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:31.655198 26361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e254d6d983441f9baf9ac19db0cea0e, State: Initialized, Role: FOLLOWER
I20260807 22:43:31.655817 26361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } }
I20260807 22:43:31.656098 26361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:31.656217 26361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:31.656358 26361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:31.659264 26361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } }
I20260807 22:43:31.659857 26361 leader_election.cc:302] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e254d6d983441f9baf9ac19db0cea0e; no voters:
I20260807 22:43:31.660606 26361 leader_election.cc:288] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:31.660826 26366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:31.661757 26366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [term 1 LEADER]: Becoming Leader. State: Replica: 7e254d6d983441f9baf9ac19db0cea0e, State: Running, Role: LEADER
I20260807 22:43:31.662473 26366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } }
I20260807 22:43:31.663127 26361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:31.665421 26367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e254d6d983441f9baf9ac19db0cea0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } } }
I20260807 22:43:31.665865 26367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:31.665868 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e254d6d983441f9baf9ac19db0cea0e. Latest consensus state: current_term: 1 leader_uuid: "7e254d6d983441f9baf9ac19db0cea0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e254d6d983441f9baf9ac19db0cea0e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 38589 } } }
I20260807 22:43:31.666747 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:31.668294 26373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:31.678774 26373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:31.691246 26373 catalog_manager.cc:1347] Generated new cluster ID: d3ca92a2888a4f728a2c8c98fd522f7e
I20260807 22:43:31.691391 26373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:31.708294 26373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:31.710088 26373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:31.721899 26373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e254d6d983441f9baf9ac19db0cea0e: Generated new TSK 0
I20260807 22:43:31.723001 26373 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:43:31.761343 26392 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 22:43:31.761812 26392 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:43:31.763005 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:31.739597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.20.10.190:38589)
0807 22:43:31.739800 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:31.739805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:31.740203 (+ 398us) server_negotiation.cc:407] Connection header received
0807 22:43:31.740472 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:31.740479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:31.740592 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:31.740811 (+ 219us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:31.741429 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.742799 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.744696 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.745563 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.745773 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.746475 (+ 702us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:31.762135 (+ 15660us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:43:31.762600 (+ 465us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 22:43:31.763322 26364 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 22:43:31.763376 26391 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:31.739345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:38589 (local address 127.0.0.1:45316)
0807 22:43:31.740061 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:31.740079 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:31.740233 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:31.740855 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:31.740860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:31.740873 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:31.741227 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.741237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.743008 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.743013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.744484 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.744502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.745834 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.745840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.746002 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.746046 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:31.762503 (+ 16457us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:43:31.762892 (+ 389us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.10.190:38589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":407,"threads_started":1}
I20260807 22:43:31.764266 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26288
I20260807 22:43:31.766059 26356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:31.864895 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26288
2026-08-07T22:43:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (872 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 22:43:31.928537 20522 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:31.930788 20522 webserver.cc:460] Webserver started at https://127.0.0.1:34533/ using document root <none> and password file <none>
I20260807 22:43:31.931077 20522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:31.931900 20522 webserver.cc:460] Webserver started at http://127.0.0.1:38729/ using document root <none> and password file <none>
2026-08-07T22:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:31Z Disabled control of system clock
I20260807 22:43:31.975836 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35613
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35613
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34533/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:43:32.378944 26400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:32.379329 26400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:32.379504 26400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:32.387532 26400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:32.387705 26400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:32.387805 26400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:32.387920 26400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:32.388012 26400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:32.388125 26400 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:32.388208 26400 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34533/jwks.json
I20260807 22:43:32.398561 26400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34533/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:32.400125 26400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:32.402308 26400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:32.416983 26405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:32.416934 26406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:32.416885 26409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:32.417925 26400 server_base.cc:1034] running on GCE node
I20260807 22:43:32.418975 26400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:32.420749 26400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:32.422086 26400 hybrid_clock.cc:648] HybridClock initialized: now 1786142612422078 us; error 139 us; skew 500 ppm
I20260807 22:43:32.422595 26400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:32.425648 26400 webserver.cc:460] Webserver started at http://127.20.10.190:34805/ using document root <none> and password file <none>
I20260807 22:43:32.426484 26400 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:32.426640 26400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:32.427050 26400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:32.430472 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "5fb687d6ae364d4992a97b9d6427364e"
format_stamp: "Formatted at 2026-08-07 22:43:32 on dist-test-slave-csg5"
server_key: "8ca18c7f4c7e0c941bbb8be906329e71"
server_key_iv: "a846ceec81649d213524cce82cb28cc9"
server_key_version: "encryptionkey@0"
I20260807 22:43:32.431591 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "5fb687d6ae364d4992a97b9d6427364e"
format_stamp: "Formatted at 2026-08-07 22:43:32 on dist-test-slave-csg5"
server_key: "8ca18c7f4c7e0c941bbb8be906329e71"
server_key_iv: "a846ceec81649d213524cce82cb28cc9"
server_key_version: "encryptionkey@0"
I20260807 22:43:32.438025 26400 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:43:32.442570 26416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:32.444535 26400 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:32.444825 26400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "5fb687d6ae364d4992a97b9d6427364e"
format_stamp: "Formatted at 2026-08-07 22:43:32 on dist-test-slave-csg5"
server_key: "8ca18c7f4c7e0c941bbb8be906329e71"
server_key_iv: "a846ceec81649d213524cce82cb28cc9"
server_key_version: "encryptionkey@0"
I20260807 22:43:32.445163 26400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:32.469275 26400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:32.470374 26400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:32.470808 26400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:32.495103 26400 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:35613
I20260807 22:43:32.495115 26474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:35613 every 8 connection(s)
I20260807 22:43:32.496896 26400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:32.497499 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26400
I20260807 22:43:32.497763 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:32.498755 20522 external_mini_cluster.cc:1428] Setting key a68ba655665426be3191a1c32c18b45b
I20260807 22:43:32.502123 26475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:32.515750 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:32.500956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.20.10.190:35613)
0807 22:43:32.502066 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:32.502078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:32.502123 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:43:32.503881 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:32.503905 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:32.504379 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:32.504732 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:32.505540 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.507173 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.509502 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.510476 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.510769 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.512565 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:32.512575 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:32.512593 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:32.513391 (+ 798us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:32.513494 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:32.513521 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:32.513648 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:32.513941 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 22:43:32.514477 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":418,"threads_started":1}
I20260807 22:43:32.517527 26475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e: Bootstrap starting.
I20260807 22:43:32.520717 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:32.521867 26475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:32.523264 26475 log.cc:824] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:32.527141 26475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e: No bootstrap required, opened a new log
I20260807 22:43:32.537030 26475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } }
I20260807 22:43:32.537603 26475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:32.537791 26475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb687d6ae364d4992a97b9d6427364e, State: Initialized, Role: FOLLOWER
I20260807 22:43:32.538594 26475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } }
I20260807 22:43:32.539021 26475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:32.539198 26475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:32.539422 26475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:32.542059 26475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } }
I20260807 22:43:32.542758 26475 leader_election.cc:302] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fb687d6ae364d4992a97b9d6427364e; no voters:
I20260807 22:43:32.543502 26475 leader_election.cc:288] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:32.543692 26481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:32.545533 26481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [term 1 LEADER]: Becoming Leader. State: Replica: 5fb687d6ae364d4992a97b9d6427364e, State: Running, Role: LEADER
I20260807 22:43:32.546159 26481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } }
I20260807 22:43:32.546380 26475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:32.549888 26483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fb687d6ae364d4992a97b9d6427364e. Latest consensus state: current_term: 1 leader_uuid: "5fb687d6ae364d4992a97b9d6427364e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } } }
I20260807 22:43:32.550376 26482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fb687d6ae364d4992a97b9d6427364e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb687d6ae364d4992a97b9d6427364e" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35613 } } }
I20260807 22:43:32.550736 26483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:32.550807 26482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:32.553635 26491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:32.560225 26491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:32.571633 26491 catalog_manager.cc:1347] Generated new cluster ID: 88588fbea2524afab164d9514662c1ab
I20260807 22:43:32.571821 26491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:32.599656 26491 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:32.601251 26491 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:32.608744 26491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fb687d6ae364d4992a97b9d6427364e: Generated new TSK 0
I20260807 22:43:32.609866 26491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:32.636981 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:32.626968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.20.10.190:35613)
0807 22:43:32.627197 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:32.627204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:32.627710 (+ 506us) server_negotiation.cc:407] Connection header received
0807 22:43:32.627970 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:32.627976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:32.628078 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:32.628292 (+ 214us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:32.628813 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.630013 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.631897 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.632671 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.632812 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.633618 (+ 806us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:32.636262 (+ 2644us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34533/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:43:32.636650 (+ 388us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41432: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34533/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 22:43:32.637141 26479 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41432: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34533/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 22:43:32.639489 26505 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:32.626791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35613 (local address 127.0.0.1:41432)
0807 22:43:32.627574 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:32.627597 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:32.627742 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:32.628318 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:32.628321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:32.628333 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:32.628644 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.628654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.630209 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.630214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.631695 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.631713 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.632946 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.632952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.633195 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.633247 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:32.636550 (+ 3303us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34533/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:43:32.638917 (+ 2367us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.10.190:35613: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34533/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":1840,"thread_start_us":419,"threads_started":1}
I20260807 22:43:32.640774 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26400
I20260807 22:43:32.642465 26470 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 22:43:32.650933 26506 webserver.cc:533] Webserver: sslize failed:
E20260807 22:43:32.651083 26506 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:43:32.743666 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26400
2026-08-07T22:43:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (878 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T22:43:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:32Z Disabled control of system clock
I20260807 22:43:32.846113 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44763
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:43699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:44763
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/testchainca.cert with env {}
W20260807 22:43:33.240844 26511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:33.241226 26511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:33.241333 26511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:33.250133 26511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:33.250281 26511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:33.250324 26511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:33.250389 26511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:33.250437 26511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:33.250497 26511 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:33.250540 26511 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 22:43:33.261373 26511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:44763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:33.262877 26511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:33.264237 26511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:33.279958 26519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:33.280503 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:33.280721 26516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:33.282323 26511 server_base.cc:1034] running on GCE node
I20260807 22:43:33.283372 26511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:33.285068 26511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:33.286341 26511 hybrid_clock.cc:648] HybridClock initialized: now 1786142613286266 us; error 61 us; skew 500 ppm
I20260807 22:43:33.286893 26511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:33.289942 26511 webserver.cc:460] Webserver started at http://127.20.10.190:46167/ using document root <none> and password file <none>
I20260807 22:43:33.290807 26511 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:33.290916 26511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:33.291287 26511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:33.294492 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "577e8370aeb54e9ca2103f420ec818af"
format_stamp: "Formatted at 2026-08-07 22:43:33 on dist-test-slave-csg5"
server_key: "a3a668a2337a02b7b694f107c770b947"
server_key_iv: "ca7e3eb1a11614b5a49fbe4f05a53dcd"
server_key_version: "encryptionkey@0"
I20260807 22:43:33.295485 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "577e8370aeb54e9ca2103f420ec818af"
format_stamp: "Formatted at 2026-08-07 22:43:33 on dist-test-slave-csg5"
server_key: "a3a668a2337a02b7b694f107c770b947"
server_key_iv: "ca7e3eb1a11614b5a49fbe4f05a53dcd"
server_key_version: "encryptionkey@0"
I20260807 22:43:33.301606 26511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:33.306118 26525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:33.307883 26511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:33.308123 26511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "577e8370aeb54e9ca2103f420ec818af"
format_stamp: "Formatted at 2026-08-07 22:43:33 on dist-test-slave-csg5"
server_key: "a3a668a2337a02b7b694f107c770b947"
server_key_iv: "ca7e3eb1a11614b5a49fbe4f05a53dcd"
server_key_version: "encryptionkey@0"
I20260807 22:43:33.308408 26511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:33.329988 26511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:33.331095 26511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:33.331444 26511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:33.354427 26511 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:44763
I20260807 22:43:33.354458 26583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:44763 every 8 connection(s)
I20260807 22:43:33.356026 26511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:33.360261 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:33.365617 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26511
I20260807 22:43:33.365869 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:33.366755 20522 external_mini_cluster.cc:1428] Setting key 898c42881950289d9cbedb2ded5a936d
I20260807 22:43:33.373623 26584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af: Bootstrap starting.
I20260807 22:43:33.377249 26584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:33.378528 26584 log.cc:824] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:33.382922 26584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af: No bootstrap required, opened a new log
I20260807 22:43:33.382958 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:33.368969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.20.10.190:44763)
0807 22:43:33.370352 (+ 1383us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:33.370366 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:33.370421 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:43:33.372472 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:33.372502 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:33.373047 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:33.373504 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:33.374215 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.375675 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.377055 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.377965 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.378213 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.379808 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:33.379820 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:33.379837 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:33.380594 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:33.380711 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:33.380742 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:33.380873 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:33.381169 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:43:33.381545 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":478,"threads_started":1}
I20260807 22:43:33.388031 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:33.392227 26584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } }
I20260807 22:43:33.392741 26584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:33.392915 26584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577e8370aeb54e9ca2103f420ec818af, State: Initialized, Role: FOLLOWER
I20260807 22:43:33.393730 26584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } }
I20260807 22:43:33.394012 26584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:33.394207 26584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:33.394418 26584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:33.397071 26584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } }
I20260807 22:43:33.397734 26584 leader_election.cc:302] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 577e8370aeb54e9ca2103f420ec818af; no voters:
I20260807 22:43:33.398772 26584 leader_election.cc:288] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:33.398895 26590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:33.399854 26590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [term 1 LEADER]: Becoming Leader. State: Replica: 577e8370aeb54e9ca2103f420ec818af, State: Running, Role: LEADER
I20260807 22:43:33.400538 26590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } }
I20260807 22:43:33.401234 26584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:33.403499 26592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 577e8370aeb54e9ca2103f420ec818af. Latest consensus state: current_term: 1 leader_uuid: "577e8370aeb54e9ca2103f420ec818af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } } }
I20260807 22:43:33.403996 26592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:33.403993 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "577e8370aeb54e9ca2103f420ec818af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577e8370aeb54e9ca2103f420ec818af" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44763 } } }
I20260807 22:43:33.404388 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:33.405202 26595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:33.412490 26595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:33.424041 26595 catalog_manager.cc:1347] Generated new cluster ID: a92c9fdb5e8c46c0981a875d546aad2a
I20260807 22:43:33.424379 26595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:33.445524 26595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:33.447571 26595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:33.463070 26595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 577e8370aeb54e9ca2103f420ec818af: Generated new TSK 0
I20260807 22:43:33.464304 26595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:33.522629 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:33.484664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.20.10.190:44763)
0807 22:43:33.484904 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:33.484912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:33.485541 (+ 629us) server_negotiation.cc:407] Connection header received
0807 22:43:33.485861 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:33.485868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:33.485987 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:33.486192 (+ 205us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:33.486815 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.488162 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.490015 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.490877 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.491024 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.491725 (+ 701us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:33.521732 (+ 30007us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:43:33.522277 (+ 545us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32890: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 22:43:33.522820 26614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:33.484582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44763 (local address 127.0.0.1:32890)
0807 22:43:33.485386 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:33.485413 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:33.485602 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:33.486246 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:33.486251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:33.486262 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:33.486583 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.486592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.488414 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.488419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.489836 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.489852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.491106 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.491113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.491258 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.491377 (+ 119us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:33.522129 (+ 30752us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:43:33.522401 (+ 272us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.10.190:44763: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":377,"threads_started":1}
W20260807 22:43:33.522847 26588 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32890: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260807 22:43:33.524752 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26511
I20260807 22:43:33.526141 26578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:33.630075 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26511
2026-08-07T22:43:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (864 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (472 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (480 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:34 dist-test-slave-csg5 krb5kdc[26632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:34 dist-test-slave-csg5 krb5kdc[26632](info): set up 2 sockets
Aug 07 22:43:34 dist-test-slave-csg5 krb5kdc[26632](info): commencing operation
krb5kdc: starting...
W20260807 22:43:35.813735 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:35 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:43:36.109855 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45595
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:45595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:36.497608 26645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:36.497982 26645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:36.498147 26645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:36.506138 26645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:36.506294 26645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:36.506345 26645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:36.506438 26645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:36.506511 26645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:36.517195 26645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:45595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:36.518747 26645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:36.520314 26645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:36.533744 26650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:36.533645 26653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:36.534036 26651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:36.534873 26645 server_base.cc:1034] running on GCE node
I20260807 22:43:36.535760 26645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:36.537550 26645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:36.538830 26645 hybrid_clock.cc:648] HybridClock initialized: now 1786142616538767 us; error 43 us; skew 500 ppm
I20260807 22:43:36.539207 26645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:36 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142616, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:36.542923 26645 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:43:36.544697 26645 webserver.cc:460] Webserver started at http://127.20.10.190:41505/ using document root <none> and password file <none>
I20260807 22:43:36.545476 26645 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:36.545595 26645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:36.545893 26645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:36.549321 26645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-csg5"
server_key: "ee679b0299bc4cf7342c3a1d5016d135"
server_key_iv: "4642871dbe2c52c48c8153b82bfe6532"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.550303 26645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-csg5"
server_key: "ee679b0299bc4cf7342c3a1d5016d135"
server_key_iv: "4642871dbe2c52c48c8153b82bfe6532"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.557178 26645 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:36.562004 26660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:36.564101 26645 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:43:36.564358 26645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-csg5"
server_key: "ee679b0299bc4cf7342c3a1d5016d135"
server_key_iv: "4642871dbe2c52c48c8153b82bfe6532"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.564742 26645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:36.587946 26645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:36.593772 26645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:36.594187 26645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:36.618556 26645 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:45595
I20260807 22:43:36.618577 26714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:45595 every 8 connection(s)
I20260807 22:43:36.620244 26645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:36.622228 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26645
I20260807 22:43:36.622535 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:36.623577 20522 external_mini_cluster.cc:1428] Setting key c44db128b39666dd1e0610377a3cfb1f
I20260807 22:43:36.626405 26715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:43:36 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:36.643494 26715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f: Bootstrap starting.
I20260807 22:43:36.648471 26715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:36.650002 26715 log.cc:824] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:36.653887 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:36.626086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.20.10.190:45595)
0807 22:43:36.627076 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:36.627089 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:36.627142 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:43:36.628982 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:36.629006 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:36.629452 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:36.629834 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:36.632104 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:36.633678 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:36.635887 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:36.636962 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:36.637139 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:36.643026 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:36.643043 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:36.643069 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:36.643153 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:36.647954 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:36.649077 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:36.649088 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:36.649098 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:36.649358 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:36.649859 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:36.649865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:36.649868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:36.650792 (+ 924us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:36.651126 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:36.651811 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 22:43:36.652341 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":521,"threads_started":1}
I20260807 22:43:36.654115 26715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f: No bootstrap required, opened a new log
I20260807 22:43:36.663092 26715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } }
I20260807 22:43:36.663507 26715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:36.663587 26715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd2f91a9bcd49d7abd5f4288e46c39f, State: Initialized, Role: FOLLOWER
I20260807 22:43:36.664193 26715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } }
I20260807 22:43:36.664434 26715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:36.664546 26715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:36.664680 26715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:36.667474 26715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } }
I20260807 22:43:36.668021 26715 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd2f91a9bcd49d7abd5f4288e46c39f; no voters:
I20260807 22:43:36.668637 26715 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:36.668845 26722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:36.669623 26722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [term 1 LEADER]: Becoming Leader. State: Replica: 1dd2f91a9bcd49d7abd5f4288e46c39f, State: Running, Role: LEADER
I20260807 22:43:36.670266 26722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } }
I20260807 22:43:36.671159 26715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:36.673524 26723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } } }
I20260807 22:43:36.674460 26723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:36.673975 26724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd2f91a9bcd49d7abd5f4288e46c39f. Latest consensus state: current_term: 1 leader_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd2f91a9bcd49d7abd5f4288e46c39f" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45595 } } }
I20260807 22:43:36.674962 26724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:36.675935 26731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:36.683418 26731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:36.696936 26731 catalog_manager.cc:1347] Generated new cluster ID: ea0462e375094c67adf7e69fd46be5ba
I20260807 22:43:36.697171 26731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:36.712729 26731 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:36.714912 26731 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:36.725800 26731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd2f91a9bcd49d7abd5f4288e46c39f: Generated new TSK 0
I20260807 22:43:36.727185 26731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:43:36.803586 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:37.191742 26746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:37.192098 26746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:37.192214 26746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:37.200284 26746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:37.200445 26746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:37.200580 26746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:37.211820 26746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:37.213815 26746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:37.216065 26746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:37.232998 26751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.233131 26754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.233031 26752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:37.234475 26746 server_base.cc:1034] running on GCE node
I20260807 22:43:37.235262 26746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:37.236459 26746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:37.237715 26746 hybrid_clock.cc:648] HybridClock initialized: now 1786142617237637 us; error 68 us; skew 500 ppm
I20260807 22:43:37.238147 26746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:37 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:37.242796 26746 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:43:37.245052 26746 webserver.cc:460] Webserver started at http://127.20.10.129:40909/ using document root <none> and password file <none>
I20260807 22:43:37.245970 26746 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:37.246132 26746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:37.246532 26746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:37.249995 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "9e98369266ca406a978e4f89aa8bffda"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "9e190dc76f2fc3a6db107cb7243f57bf"
server_key_iv: "18c9ab0bf1d16b811d9fb27c24afe195"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.251044 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "9e98369266ca406a978e4f89aa8bffda"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "9e190dc76f2fc3a6db107cb7243f57bf"
server_key_iv: "18c9ab0bf1d16b811d9fb27c24afe195"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.257486 26746 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:43:37.261994 26761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.263947 26746 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:37.264159 26746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "9e98369266ca406a978e4f89aa8bffda"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "9e190dc76f2fc3a6db107cb7243f57bf"
server_key_iv: "18c9ab0bf1d16b811d9fb27c24afe195"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.264394 26746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:37.278136 26746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:37.284291 26746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:37.284694 26746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:37.285995 26746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:37.287768 26746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:37.287856 26746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.288003 26746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:37.288089 26746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.330955 26746 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:35409
I20260807 22:43:37.331002 26882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:35409 every 8 connection(s)
I20260807 22:43:37.332386 26746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:37.334015 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26746
I20260807 22:43:37.334272 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:37.335104 20522 external_mini_cluster.cc:1428] Setting key b43327ed4505e98cf13a569d0e157d95
Aug 07 22:43:37 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:37.359717 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.335743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:41699 (local address 127.20.10.190:45595)
0807 22:43:37.336339 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:37.336346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:37.337559 (+ 1213us) server_negotiation.cc:407] Connection header received
0807 22:43:37.341100 (+ 3541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:37.341104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:37.341182 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:37.341283 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:37.343927 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.344700 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.347719 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.348300 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.348435 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.352956 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:37.352962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:37.352966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:37.353013 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:37.356082 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.357027 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.357032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.357034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.357170 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.357657 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.357662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.357664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.357958 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:37.358127 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:37.359062 (+ 935us) server_negotiation.cc:299] Negotiation successful
0807 22:43:37.359341 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":281,"threads_started":1}
I20260807 22:43:37.360775 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.336406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45595 (local address 127.20.10.129:41699)
0807 22:43:37.337359 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:37.337421 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:37.340708 (+ 3287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:37.341633 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:37.341650 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:37.342621 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:37.343660 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.343708 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.344901 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.344911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.347492 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.347523 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.348630 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.348641 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.349000 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.349838 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:37.349897 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:37.352616 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:37.356290 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.356304 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.356325 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.356863 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.357293 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.357298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.357301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.357516 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.358096 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:37.358105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:37.358491 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 22:43:37.358932 (+ 441us) client_negotiation.cc:240] Negotiation successful
0807 22:43:37.359537 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":363,"threads_started":1}
I20260807 22:43:37.362773 26883 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45595
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:43:37.363144 26883 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:37.363932 26883 heartbeater.cc:511] Master 127.20.10.190:45595 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:43:37.366714 26679 ts_manager.cc:194] Registered new tserver with Master: 9e98369266ca406a978e4f89aa8bffda (127.20.10.129:35409)
I20260807 22:43:37.368436 26679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:41699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:43:37.404034 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:37.779814 26889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:37.780107 26889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:37.780269 26889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:37.788197 26889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:37.788331 26889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:37.788518 26889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:37.798986 26889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:37.800434 26889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:37.802001 26889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:37.818493 26894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.818681 26895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.818812 26897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:37.820120 26889 server_base.cc:1034] running on GCE node
I20260807 22:43:37.820967 26889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:37.822222 26889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:37.823521 26889 hybrid_clock.cc:648] HybridClock initialized: now 1786142617823466 us; error 47 us; skew 500 ppm
I20260807 22:43:37.823928 26889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:37 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:37.827538 26889 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:43:37.829262 26889 webserver.cc:460] Webserver started at http://127.20.10.130:33267/ using document root <none> and password file <none>
I20260807 22:43:37.830080 26889 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:37.830255 26889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:37.830588 26889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:37.833737 26889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "6b92a8cb1bf14b6aacd8f9b2f445978f"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "3aab79430f7dabb24e166f4c51b7101c"
server_key_iv: "19d636324d0ceaf381d9b3d70d5e172c"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.834700 26889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "6b92a8cb1bf14b6aacd8f9b2f445978f"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "3aab79430f7dabb24e166f4c51b7101c"
server_key_iv: "19d636324d0ceaf381d9b3d70d5e172c"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.840288 26889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:43:37.844480 26905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.846130 26889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:43:37.846350 26889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "6b92a8cb1bf14b6aacd8f9b2f445978f"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-csg5"
server_key: "3aab79430f7dabb24e166f4c51b7101c"
server_key_iv: "19d636324d0ceaf381d9b3d70d5e172c"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.846590 26889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:37.872529 26889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:37.878104 26889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:37.878422 26889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:37.879714 26889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:37.881196 26889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:37.881273 26889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.881362 26889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:37.881453 26889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.917654 26889 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:33575
I20260807 22:43:37.917740 27029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:33575 every 8 connection(s)
I20260807 22:43:37.919037 26889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:37.924767 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 26889
I20260807 22:43:37.925007 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:37.925630 20522 external_mini_cluster.cc:1428] Setting key 1081536925578198643c45667b9d3a36
Aug 07 22:43:37 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:37.943920 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.921896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:54673 (local address 127.20.10.190:45595)
0807 22:43:37.922437 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:37.922444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:37.923472 (+ 1028us) server_negotiation.cc:407] Connection header received
0807 22:43:37.925840 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:37.925847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:37.925962 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:37.926099 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:37.928199 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.929049 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.931284 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.931789 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.931944 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.936451 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:37.936461 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:37.936468 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:37.936521 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:37.939667 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.940924 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.940930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.940932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.941096 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.941822 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.941826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.941829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.942142 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:37.942312 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:37.943423 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0807 22:43:37.943631 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
I20260807 22:43:37.945830 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.922370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45595 (local address 127.20.10.130:54673)
0807 22:43:37.923285 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:37.923337 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:37.925439 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:37.926423 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:37.926436 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:37.927206 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:37.928001 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.928023 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.929248 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.929252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.931078 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.931093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.932084 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.932093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.932368 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.933140 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:37.933164 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:37.936108 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:37.939881 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.939895 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.939920 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.940714 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.941259 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.941267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.941273 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.941600 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.942317 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:37.942328 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:37.942943 (+ 615us) client_negotiation.cc:769] Sending connection context
0807 22:43:37.943289 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 22:43:37.943977 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":385,"threads_started":1}
I20260807 22:43:37.947909 27030 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45595
I20260807 22:43:37.948299 27030 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:37.949147 27030 heartbeater.cc:511] Master 127.20.10.190:45595 requested a full tablet report, sending...
I20260807 22:43:37.951093 26679 ts_manager.cc:194] Registered new tserver with Master: 6b92a8cb1bf14b6aacd8f9b2f445978f (127.20.10.130:33575)
I20260807 22:43:37.952044 26679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:54673
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:43:37.995568 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:38.372738 26883 heartbeater.cc:503] Master 127.20.10.190:45595 was elected leader, sending a full tablet report...
W20260807 22:43:38.384827 27036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:38.385239 27036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:38.385445 27036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:38.394073 27036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:38.394256 27036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:38.394448 27036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:38.405320 27036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:38.406895 27036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:38.408326 27036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:38.424602 27041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:38.424525 27042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:38.424572 27045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:38.425633 27036 server_base.cc:1034] running on GCE node
I20260807 22:43:38.426433 27036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:38.427618 27036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:38.428891 27036 hybrid_clock.cc:648] HybridClock initialized: now 1786142618428819 us; error 77 us; skew 500 ppm
I20260807 22:43:38.429276 27036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.433035 27036 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:43:38.434965 27036 webserver.cc:460] Webserver started at http://127.20.10.131:42655/ using document root <none> and password file <none>
I20260807 22:43:38.435755 27036 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:38.435916 27036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:38.436265 27036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:38.439584 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "da2b426f02d1452dbb64244f7bdc0c27"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-csg5"
server_key: "519b88debe69e3650eefdd273877e6b1"
server_key_iv: "7393bb30e44791a586007f72e929d454"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.440474 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "da2b426f02d1452dbb64244f7bdc0c27"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-csg5"
server_key: "519b88debe69e3650eefdd273877e6b1"
server_key_iv: "7393bb30e44791a586007f72e929d454"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.446777 27036 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260807 22:43:38.451107 27053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.453038 27036 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 22:43:38.453279 27036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "da2b426f02d1452dbb64244f7bdc0c27"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-csg5"
server_key: "519b88debe69e3650eefdd273877e6b1"
server_key_iv: "7393bb30e44791a586007f72e929d454"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.453569 27036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:38.500164 27036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:38.505441 27036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:38.505883 27036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:38.507066 27036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:38.508713 27036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:38.508832 27036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.508987 27036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:38.509059 27036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.548938 27036 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:45859
I20260807 22:43:38.548996 27179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:45859 every 8 connection(s)
I20260807 22:43:38.550516 27036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:38.557587 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27036
I20260807 22:43:38.557837 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:38.558451 20522 external_mini_cluster.cc:1428] Setting key 7bb1a2f49443c94f24c5f70d125dcc9b
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:43:38.577549 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.553917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:37279 (local address 127.20.10.190:45595)
0807 22:43:38.554636 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.554641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.555570 (+ 929us) server_negotiation.cc:407] Connection header received
0807 22:43:38.558888 (+ 3318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.558896 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.559082 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.559209 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.561242 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.562290 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.564031 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.564494 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.564614 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.569072 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.569079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.569083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.569209 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.573225 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.574549 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.574555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.574558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.574790 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.575504 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.575510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.575513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.575929 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.576096 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.576891 (+ 795us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.577144 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":457,"threads_started":1}
I20260807 22:43:38.578706 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.554407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45595 (local address 127.20.10.131:37279)
0807 22:43:38.555367 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.555421 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.558288 (+ 2867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.559549 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.559563 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.560297 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:38.561063 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.561087 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.562466 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.562470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.563850 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.563870 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.564749 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.564755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.564969 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.565908 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:38.565935 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:38.568769 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:38.573437 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.573447 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.573465 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.574356 (+ 891us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.574921 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.574928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.574932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.575153 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.576079 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:38.576086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:38.576484 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.576777 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.577410 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":468,"threads_started":1}
I20260807 22:43:38.580646 27180 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45595
I20260807 22:43:38.581063 27180 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:38.581911 27180 heartbeater.cc:511] Master 127.20.10.190:45595 requested a full tablet report, sending...
I20260807 22:43:38.583671 26679 ts_manager.cc:194] Registered new tserver with Master: da2b426f02d1452dbb64244f7bdc0c27 (127.20.10.131:45859)
I20260807 22:43:38.584625 26679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:37279
I20260807 22:43:38.588814 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/test-admin.keytab.
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.627434 20522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 22:43:38.633644 27190 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:38.630807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.20.10.129:35409)
0807 22:43:38.631625 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.631633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.631660 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:38.631723 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.631729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.631946 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.632074 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.633162 (+ 1088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42780: BlockingRecv error: recv got EOF from 127.0.0.1:42780 (error 108)
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":516,"threads_started":1}
W20260807 22:43:38.634001 20522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.636155 20522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.637771 20522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:38.656086 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.639926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.20.10.129:35409)
0807 22:43:38.640191 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.640198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.640524 (+ 326us) server_negotiation.cc:407] Connection header received
0807 22:43:38.640766 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.640772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.640852 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.640966 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.642064 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.643144 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.644450 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.645050 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.645204 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.649324 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.649340 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.649356 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.649411 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.652834 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.653757 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.653765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.653767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.653917 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.654293 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.654353 (+ 60us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.654356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.654840 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.655086 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.655500 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.655766 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260807 22:43:38.663223 27190 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:38.661159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.20.10.129:35409)
0807 22:43:38.661443 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.661450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.661595 (+ 145us) server_negotiation.cc:407] Connection header received
0807 22:43:38.661793 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.661799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.661883 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.661975 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.662901 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42800: BlockingRecv error: recv got EOF from 127.0.0.1:42800 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260807 22:43:38.663525 20522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.665529 20522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.667126 20522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:38.686120 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.668947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.20.10.129:35409)
0807 22:43:38.669172 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.669177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.669299 (+ 122us) server_negotiation.cc:407] Connection header received
0807 22:43:38.669403 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.669408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.669514 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.669615 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.670845 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.671760 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.673138 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.673654 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.673806 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.678846 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.678856 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.678861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.678931 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.682179 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.683435 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.683441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.683447 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.683635 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.684284 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.684289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.684291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.684642 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.684904 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.685541 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.685862 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 22:43:38.692809 27190 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:38.690021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.20.10.129:35409)
0807 22:43:38.690330 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.690341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.690845 (+ 504us) server_negotiation.cc:407] Connection header received
0807 22:43:38.691043 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.691047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.691141 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.691250 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.692519 (+ 1269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42814: BlockingRecv error: recv got EOF from 127.0.0.1:42814 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 22:43:38.693512 20522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142560216906-20522-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.695559 20522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.697250 20522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:43:38.717054 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.699126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.20.10.129:35409)
0807 22:43:38.699415 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.699421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.699595 (+ 174us) server_negotiation.cc:407] Connection header received
0807 22:43:38.699909 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.699913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.699996 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.700095 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.701389 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.702220 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.703920 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.704406 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.704537 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.709766 (+ 5229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.709773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.709776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.709827 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.712977 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.714112 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.714117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.714120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.714431 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.715092 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.715097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.715100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.715557 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.715884 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.716373 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.716740 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:43:38.719308 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26746
I20260807 22:43:38.735946 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.720205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.20.10.129:35409)
0807 22:43:38.720467 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.720474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.720787 (+ 313us) server_negotiation.cc:407] Connection header received
0807 22:43:38.720987 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.720991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.721064 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.721176 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.722394 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.723334 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.724749 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.725417 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.725547 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.728352 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.728358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.728362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.728408 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.732230 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.733131 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.733136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.733139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.733393 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.733938 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.733943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.733945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.734407 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.734772 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.735150 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.735526 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:43:38.736652 26878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:38.826098 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26746
I20260807 22:43:38.851038 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26889
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:43:38.869154 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.851972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.20.10.130:33575)
0807 22:43:38.852594 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.852601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.852627 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:38.852690 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.852695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.852909 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.853084 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.854171 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.855342 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.856667 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.857248 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.857422 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.862253 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.862271 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.862288 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.862333 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.865825 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.866755 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.866767 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.866770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.866943 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.867370 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.867375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.867379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.867788 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.868131 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.868496 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.868793 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":276,"threads_started":1}
I20260807 22:43:38.870538 27024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:38.959585 27030 heartbeater.cc:503] Master 127.20.10.190:45595 was elected leader, sending a full tablet report...
I20260807 22:43:38.961328 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26889
I20260807 22:43:38.984153 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27036
W20260807 22:43:38.984025 26663 connection.cc:535] server connection from 127.20.10.130:54673 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Aug 07 22:43:38 dist-test-slave-csg5 krb5kdc[26632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:43:39.004117 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.985350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.20.10.131:45859)
0807 22:43:38.985960 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.985967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.986008 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:43:38.986077 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.986083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.986512 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.986726 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.987947 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.988986 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.990334 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.991100 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.991265 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.995965 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.995983 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.996000 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.996043 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:39.000387 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:39.001571 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:39.001580 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:39.001582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:39.001760 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:39.002217 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:39.002225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:39.002229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:39.002764 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:39.003126 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:39.003458 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 22:43:39.003767 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":313,"threads_started":1}
I20260807 22:43:39.005820 27174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:39.095744 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27036
I20260807 22:43:39.119151 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 26645
I20260807 22:43:39.120688 26710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:39.238375 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 26645
2026-08-07T22:43:39Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4665 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:39 dist-test-slave-csg5 krb5kdc[27210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:39 dist-test-slave-csg5 krb5kdc[27210](info): set up 2 sockets
Aug 07 22:43:39 dist-test-slave-csg5 krb5kdc[27210](info): commencing operation
krb5kdc: starting...
W20260807 22:43:40.521283 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:40 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab.
Entry for principal oryx/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab.
I20260807 22:43:40.820492 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:43009
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab
--principal=oryx/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:43009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:41.195708 27223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:41.196050 27223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:41.196283 27223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:41.205509 27223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:41.205657 27223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:41.205705 27223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:41.205783 27223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:41.205830 27223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:41.216936 27223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:43009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab
--principal=oryx/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:43009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:41.218379 27223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:41.219882 27223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:41.233906 27230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:41.234746 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:41.234964 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:41.235390 27223 server_base.cc:1034] running on GCE node
I20260807 22:43:41.236311 27223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:41.238049 27223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:41.239442 27223 hybrid_clock.cc:648] HybridClock initialized: now 1786142621239232 us; error 82 us; skew 500 ppm
I20260807 22:43:41.239859 27223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:41 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142621, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:41.248706 27223 init.cc:377] Logged in from keytab as oryx/127.20.10.190@KRBTEST.COM (short username oryx)
I20260807 22:43:41.250629 27223 webserver.cc:460] Webserver started at http://127.20.10.190:40749/ using document root <none> and password file <none>
I20260807 22:43:41.251480 27223 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:41.251644 27223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:41.251986 27223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:41.255445 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6f36bc52d5b1432283883efed09e4aea"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "6b6cc09b2f4b3ae89795c803eebe0d9d"
server_key_iv: "a9faa9ab5fa9b47de5f5b424d0709b20"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.256328 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6f36bc52d5b1432283883efed09e4aea"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "6b6cc09b2f4b3ae89795c803eebe0d9d"
server_key_iv: "a9faa9ab5fa9b47de5f5b424d0709b20"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.262125 27223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:43:41.266271 27240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:41.268107 27223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:41.268452 27223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6f36bc52d5b1432283883efed09e4aea"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "6b6cc09b2f4b3ae89795c803eebe0d9d"
server_key_iv: "a9faa9ab5fa9b47de5f5b424d0709b20"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.268705 27223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:41.288086 27223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:41.294425 27223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:41.294899 27223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:41.318327 27223 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:43009
I20260807 22:43:41.318432 27297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:43009 every 8 connection(s)
I20260807 22:43:41.320405 27223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:41.321554 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27223
I20260807 22:43:41.321841 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:41.322587 20522 external_mini_cluster.cc:1428] Setting key 4146eab1056110c2bdbfe229c49427b7
I20260807 22:43:41.325784 27298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:43:41 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:41.341522 27298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Bootstrap starting.
I20260807 22:43:41.346637 27298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:41.348201 27298 log.cc:824] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:41.352519 27298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: No bootstrap required, opened a new log
I20260807 22:43:41.352887 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:41.325229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.20.10.190:43009)
0807 22:43:41.326254 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:41.326273 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:41.326332 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:43:41.329149 (+ 2817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:41.329184 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:41.329840 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:41.330395 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:41.332270 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:41.334636 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:41.336178 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:41.337408 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:41.337630 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:41.342145 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:41.342159 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:41.342185 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:41.342242 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:41.347141 (+ 4899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:41.348188 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:41.348199 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:41.348209 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:41.348509 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:41.349107 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:41.349112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:41.349114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:41.349841 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:41.350238 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:41.350739 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 22:43:41.351312 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":402,"threads_started":1}
I20260807 22:43:41.362367 27298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } }
I20260807 22:43:41.362972 27298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:41.363137 27298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f36bc52d5b1432283883efed09e4aea, State: Initialized, Role: FOLLOWER
I20260807 22:43:41.363905 27298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } }
I20260807 22:43:41.364178 27298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:41.364382 27298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:41.364600 27298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:41.367287 27298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } }
I20260807 22:43:41.367893 27298 leader_election.cc:302] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f36bc52d5b1432283883efed09e4aea; no voters:
I20260807 22:43:41.368670 27298 leader_election.cc:288] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:41.368854 27303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:41.370303 27303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 LEADER]: Becoming Leader. State: Replica: 6f36bc52d5b1432283883efed09e4aea, State: Running, Role: LEADER
I20260807 22:43:41.371032 27303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } }
I20260807 22:43:41.371378 27298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:41.374526 27304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } }
I20260807 22:43:41.375231 27304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:41.374887 27306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f36bc52d5b1432283883efed09e4aea. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } }
I20260807 22:43:41.375718 27306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:41.379441 27313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:41.387095 27313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:41.402130 27313 catalog_manager.cc:1347] Generated new cluster ID: ace5f31a27fa4c9cb79d54f2f3af9199
I20260807 22:43:41.402387 27313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:41.414605 27313 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:41.416147 27313 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:41.425843 27313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Generated new TSK 0
I20260807 22:43:41.427141 27313 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
I20260807 22:43:41.496115 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:41.917906 27327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:41.918238 27327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:41.918409 27327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:41.926636 27327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:41.926838 27327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:41.927016 27327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:41.937669 27327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:41.939195 27327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:41.940601 27327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:41.955163 27332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:41.955076 27336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:41.955101 27333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:41.957049 27327 server_base.cc:1034] running on GCE node
I20260807 22:43:41.957813 27327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:41.959012 27327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:41.960409 27327 hybrid_clock.cc:648] HybridClock initialized: now 1786142621960319 us; error 75 us; skew 500 ppm
I20260807 22:43:41.960760 27327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:41 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142621, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:41.964406 27327 init.cc:377] Logged in from keytab as oryx/127.20.10.129@KRBTEST.COM (short username oryx)
I20260807 22:43:41.966187 27327 webserver.cc:460] Webserver started at http://127.20.10.129:43871/ using document root <none> and password file <none>
I20260807 22:43:41.967039 27327 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:41.967159 27327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:41.967427 27327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:41.970708 27327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "81ee3502329748c1af23a06553254d6e"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "7a3726d43e5fb46af62f798a7c614ab4"
server_key_iv: "dc35cc8de84a4f9c2568d3668f9b454f"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.971661 27327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "81ee3502329748c1af23a06553254d6e"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "7a3726d43e5fb46af62f798a7c614ab4"
server_key_iv: "dc35cc8de84a4f9c2568d3668f9b454f"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.977697 27327 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:43:41.981786 27344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:41.983731 27327 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:41.983939 27327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "81ee3502329748c1af23a06553254d6e"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "7a3726d43e5fb46af62f798a7c614ab4"
server_key_iv: "dc35cc8de84a4f9c2568d3668f9b454f"
server_key_version: "encryptionkey@0"
I20260807 22:43:41.984216 27327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:42.014318 27327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:42.019086 27327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:42.019433 27327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:42.020526 27327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:42.022111 27327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:42.022192 27327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:42.022347 27327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:42.022445 27327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:42.061843 27327 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41251
I20260807 22:43:42.061892 27466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41251 every 8 connection(s)
I20260807 22:43:42.063331 27327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:42.064363 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27327
I20260807 22:43:42.064615 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:42.065560 20522 external_mini_cluster.cc:1428] Setting key 501d0cfe14759e40dc0553a0564b609e
Aug 07 22:43:42 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142621, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:42.091089 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:42.066812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:49923 (local address 127.20.10.190:43009)
0807 22:43:42.067351 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:42.067358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:42.068539 (+ 1181us) server_negotiation.cc:407] Connection header received
0807 22:43:42.071631 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:42.071636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:42.071753 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:42.071858 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:42.074203 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.075103 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.077837 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.078358 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.078474 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:42.082888 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:42.082897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:42.082905 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:42.082955 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:42.086954 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:42.087893 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:42.087902 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:42.087908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:42.088075 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:42.088627 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:42.088634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:42.088641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:42.089161 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:42.089333 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:42.090240 (+ 907us) server_negotiation.cc:299] Negotiation successful
0807 22:43:42.090543 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":260,"threads_started":1}
I20260807 22:43:42.092056 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:42.067447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:49923)
0807 22:43:42.068340 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:42.068400 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:42.071284 (+ 2884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:42.072158 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:42.072171 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:42.073061 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:42.073964 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:42.073994 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.075281 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.075288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:42.077584 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:42.077604 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.078621 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.078631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:42.079032 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:42.079864 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:42.079889 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:42.082539 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:42.087156 (+ 4617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:42.087164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:42.087181 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:42.087770 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:42.088261 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:42.088265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:42.088267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:42.088488 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:42.089385 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:42.089392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:42.089829 (+ 437us) client_negotiation.cc:769] Sending connection context
0807 22:43:42.090099 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 22:43:42.090615 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":371,"threads_started":1}
I20260807 22:43:42.093720 27467 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:42.094060 27467 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:42.094973 27467 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:43:42.097388 27259 ts_manager.cc:194] Registered new tserver with Master: 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.130@KRBTEST.COM" created.
I20260807 22:43:42.099612 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:49923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
I20260807 22:43:42.136215 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:42.519420 27474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:42.519799 27474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:42.519987 27474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:42.527835 27474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:42.528023 27474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:42.528220 27474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:42.539104 27474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:42.540634 27474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:42.542171 27474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:42.559487 27480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:42.559424 27483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:42.559464 27479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:42.560357 27474 server_base.cc:1034] running on GCE node
I20260807 22:43:42.561128 27474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:42.562353 27474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:42.563611 27474 hybrid_clock.cc:648] HybridClock initialized: now 1786142622563582 us; error 75 us; skew 500 ppm
I20260807 22:43:42.564019 27474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:42 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142622, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:42.568048 27474 init.cc:377] Logged in from keytab as oryx/127.20.10.130@KRBTEST.COM (short username oryx)
I20260807 22:43:42.569909 27474 webserver.cc:460] Webserver started at http://127.20.10.130:39117/ using document root <none> and password file <none>
I20260807 22:43:42.570770 27474 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:42.570902 27474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:42.571287 27474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:42.575199 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
format_stamp: "Formatted at 2026-08-07 22:43:42 on dist-test-slave-csg5"
server_key: "62de8a889e6f26072ed9c1f609fbb8cb"
server_key_iv: "284bc1fa83ae24b7491875a496924986"
server_key_version: "encryptionkey@0"
I20260807 22:43:42.576113 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
format_stamp: "Formatted at 2026-08-07 22:43:42 on dist-test-slave-csg5"
server_key: "62de8a889e6f26072ed9c1f609fbb8cb"
server_key_iv: "284bc1fa83ae24b7491875a496924986"
server_key_version: "encryptionkey@0"
I20260807 22:43:42.582154 27474 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 22:43:42.586534 27490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:42.588550 27474 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:42.588773 27474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
format_stamp: "Formatted at 2026-08-07 22:43:42 on dist-test-slave-csg5"
server_key: "62de8a889e6f26072ed9c1f609fbb8cb"
server_key_iv: "284bc1fa83ae24b7491875a496924986"
server_key_version: "encryptionkey@0"
I20260807 22:43:42.589028 27474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:42.620939 27474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:42.626385 27474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:42.626717 27474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:42.627907 27474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:42.629467 27474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:42.629549 27474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:42.629707 27474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:42.629817 27474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:42.670111 27474 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:38223
I20260807 22:43:42.670312 27608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:38223 every 8 connection(s)
I20260807 22:43:42.671746 27474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:42.677029 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27474
I20260807 22:43:42.677263 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:42.677726 20522 external_mini_cluster.cc:1428] Setting key 48f4a0a2b4450c2d04f3ebdc23d192e1
Aug 07 22:43:42 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142622, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:42.696367 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:42.675060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:43911 (local address 127.20.10.190:43009)
0807 22:43:42.675599 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:42.675605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:42.676655 (+ 1050us) server_negotiation.cc:407] Connection header received
0807 22:43:42.678916 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:42.678923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:42.679036 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:42.679164 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:42.681092 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.681976 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.684131 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.684625 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.684788 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:42.689118 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:42.689124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:42.689127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:42.689168 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:42.692471 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:42.693520 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:42.693525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:42.693527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:42.693681 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:42.694278 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:42.694282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:42.694284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:42.694598 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:42.694827 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:42.695668 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 22:43:42.695951 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":298,"threads_started":1}
I20260807 22:43:42.697577 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:42.675653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:43911)
0807 22:43:42.676462 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:42.676513 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:42.678529 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:42.679446 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:42.679456 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:42.680069 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:42.680723 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:42.680744 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.682183 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.682188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:42.683946 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:42.683965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:42.684923 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:42.684932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:42.685216 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:42.686062 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:42.686088 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:42.688797 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:42.692697 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:42.692709 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:42.692739 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:42.693338 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:42.693822 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:42.693827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:42.693830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:42.694126 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:42.694814 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:42.694823 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:42.695231 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 22:43:42.695538 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 22:43:42.696220 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":321,"threads_started":1}
I20260807 22:43:42.699401 27609 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:42.699815 27609 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:42.700944 27609 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
I20260807 22:43:42.703433 27259 ts_manager.cc:194] Registered new tserver with Master: cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223)
I20260807 22:43:42.704977 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:43911
WARNING: no policy specified for oryx/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
I20260807 22:43:42.748119 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:43.103668 27467 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
W20260807 22:43:43.142086 27616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:43.142454 27616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:43.142567 27616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:43.151461 27616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:43.151600 27616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:43.151818 27616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:43.163033 27616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:43009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:43.164549 27616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:43.166036 27616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:43.180948 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:43.180896 27622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:43.180994 27624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:43.181797 27616 server_base.cc:1034] running on GCE node
I20260807 22:43:43.182484 27616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:43.183694 27616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:43.184944 27616 hybrid_clock.cc:648] HybridClock initialized: now 1786142623184899 us; error 36 us; skew 500 ppm
I20260807 22:43:43.185288 27616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142623, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:43.189007 27616 init.cc:377] Logged in from keytab as oryx/127.20.10.131@KRBTEST.COM (short username oryx)
I20260807 22:43:43.190923 27616 webserver.cc:460] Webserver started at http://127.20.10.131:33561/ using document root <none> and password file <none>
I20260807 22:43:43.191675 27616 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:43.191810 27616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:43.192152 27616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:43.195544 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "acd9d8b8ace261b9e30d4ddaaec5f91e"
server_key_iv: "d0eef07ad1bf7e5812a9745cd3f8a4db"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.196426 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "acd9d8b8ace261b9e30d4ddaaec5f91e"
server_key_iv: "d0eef07ad1bf7e5812a9745cd3f8a4db"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.202551 27616 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:43:43.206745 27632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:43.208312 27616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:43:43.208521 27616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "acd9d8b8ace261b9e30d4ddaaec5f91e"
server_key_iv: "d0eef07ad1bf7e5812a9745cd3f8a4db"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.208755 27616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:43.227700 27616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:43.232575 27616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:43.232933 27616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:43.234014 27616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:43.235579 27616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:43.235656 27616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:43.235802 27616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:43.235855 27616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:43.277761 27616 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:35225
I20260807 22:43:43.277805 27752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:35225 every 8 connection(s)
I20260807 22:43:43.279477 27616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:43.289021 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27616
I20260807 22:43:43.289383 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:43.289902 20522 external_mini_cluster.cc:1428] Setting key 86f3f29286c84b93c92767f084efd334
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142623, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:43.305140 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.282773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:35873 (local address 127.20.10.190:43009)
0807 22:43:43.283475 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:43.283481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:43.284539 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 22:43:43.286840 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:43.286846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:43.286942 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:43.287057 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:43.289391 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.290417 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.293188 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.293972 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.294120 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.298491 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:43.298497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:43.298501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:43.298539 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:43.301527 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.302388 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.302393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.302395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.302550 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.303134 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.303138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.303140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.303511 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:43.303672 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:43.304447 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 22:43:43.304653 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":314,"threads_started":1}
I20260807 22:43:43.306520 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.283214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:35873)
0807 22:43:43.284333 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:43.284388 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:43.286412 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:43.287379 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:43.287389 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:43.288228 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:43.289084 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.289155 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.290760 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.290765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.292994 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.293010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.294281 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.294291 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.294546 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.295432 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:43.295464 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:43.298176 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:43.301694 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.301703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.301719 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.302216 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.302733 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.302738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.302740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.302977 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.303655 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:43.303661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:43.304050 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 22:43:43.304312 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 22:43:43.304913 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":529,"threads_started":1}
I20260807 22:43:43.308473 27753 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:43.308869 27753 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:43.310071 27753 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
I20260807 22:43:43.312105 27259 ts_manager.cc:194] Registered new tserver with Master: 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:43.313125 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:35873
I20260807 22:43:43.320072 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.10.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.189.keytab.
Entry for principal oryx/127.20.10.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.189.keytab.
I20260807 22:43:43.388677 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:43165
--webserver_interface=127.20.10.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.189.keytab
--principal=oryx/127.20.10.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:43009,127.20.10.189:43165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:43.709462 27609 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
W20260807 22:43:43.795702 27759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:43.796092 27759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:43.796260 27759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:43.805045 27759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:43.805193 27759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:43.805282 27759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:43.805382 27759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:43.805457 27759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:43.817680 27759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:43009,127.20.10.189:43165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.189.keytab
--principal=oryx/127.20.10.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:43165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.10.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
W20260807 22:43:43.818176 27759 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.10.190:43009,127.20.10.189:43165'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 22:43:43.819545 27759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:43.821166 27759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:43.835548 27764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:43.836068 27765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:43.836095 27767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:43.837385 27759 server_base.cc:1034] running on GCE node
I20260807 22:43:43.838312 27759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:43.839998 27759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:43.841266 27759 hybrid_clock.cc:648] HybridClock initialized: now 1786142623841180 us; error 39 us; skew 500 ppm
I20260807 22:43:43.841686 27759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142623, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:43.845646 27759 init.cc:377] Logged in from keytab as oryx/127.20.10.189@KRBTEST.COM (short username oryx)
I20260807 22:43:43.847431 27759 webserver.cc:460] Webserver started at http://127.20.10.189:39123/ using document root <none> and password file <none>
I20260807 22:43:43.848227 27759 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:43.848340 27759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:43.848749 27759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:43.852216 27759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/instance:
uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "c998994a2359f865641bec3e84a2c651"
server_key_iv: "78a5ee362df99316bfc178719c44d02f"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.853098 27759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal/instance:
uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "c998994a2359f865641bec3e84a2c651"
server_key_iv: "78a5ee362df99316bfc178719c44d02f"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.859431 27759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:43:43.863520 27775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:43.865374 27759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:43:43.865581 27759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "c998994a2359f865641bec3e84a2c651"
server_key_iv: "78a5ee362df99316bfc178719c44d02f"
server_key_version: "encryptionkey@0"
I20260807 22:43:43.865849 27759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:43.882782 27759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:43.888559 27759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:43.889073 27759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:43.911051 27759 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.189:43165
I20260807 22:43:43.911093 27834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.189:43165 every 8 connection(s)
I20260807 22:43:43.912575 27759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
I20260807 22:43:43.917292 27835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:43.921046 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27759
I20260807 22:43:43.925827 27835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142623, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.189@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:43.944880 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.923535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.20.10.189:43165)
0807 22:43:43.924386 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:43.924398 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:43.924441 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:43:43.926266 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:43.926289 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:43.926791 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:43.927198 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:43.928761 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.930405 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.931881 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.932613 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.932807 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.937081 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:43.937093 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:43.937106 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:43.937151 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:43.940454 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.941306 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.941313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.941320 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.941483 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.941889 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.941901 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.941903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.942466 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:43.942831 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:43.943316 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 22:43:43.943814 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":585,"thread_start_us":351,"threads_started":1}
I20260807 22:43:43.946729 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.929187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.20.10.190:43009)
0807 22:43:43.929770 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:43.929776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:43.930469 (+ 693us) server_negotiation.cc:407] Connection header received
0807 22:43:43.930742 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:43.930747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:43.930836 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:43.930971 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:43.932360 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.933222 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.934802 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.935278 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.935439 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.939852 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:43.939858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:43.939862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:43.939907 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:43.943209 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.944203 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.944210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.944214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.944369 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.944810 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.944817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.944821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.945295 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:43.945472 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:43.945994 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 22:43:43.946268 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":297,"threads_started":1}
I20260807 22:43:43.946827 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.929219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38534)
0807 22:43:43.930292 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:43.930341 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:43.930533 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:43.931042 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:43.931050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:43.931797 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:43.932178 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.932194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.933373 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.933381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.934550 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.934568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.935544 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.935555 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.935755 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.936449 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:43.936474 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:43.939572 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:43.943368 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.943385 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.943392 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.944063 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.944501 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.944507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.944511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.944681 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.945439 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:43.945447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:43.945659 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 22:43:43.945877 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 22:43:43.946262 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":39,"thread_start_us":625,"threads_started":1}
I20260807 22:43:43.948885 27835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:43 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142623, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.189@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
I20260807 22:43:43.963181 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.949816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39914)
0807 22:43:43.950067 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:43.950092 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:43.950169 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:43.950535 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:43.950538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:43.950968 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:43.951112 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.951120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.952180 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.952187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.953227 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:43.953240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.954017 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.954027 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:43.954266 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.954987 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:43.955016 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:43.957233 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:43.960522 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.960527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.960530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.960937 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.961458 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:43.961461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:43.961463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:43.961594 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:43.962342 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:43.962347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:43.962473 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 22:43:43.962591 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 22:43:43.962930 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 22:43:43.963541 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:43.949900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.20.10.189:43165)
0807 22:43:43.950103 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:43.950108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:43.950148 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:43:43.950343 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:43.950348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:43.950410 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:43.950498 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:43.951288 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.951982 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.953373 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:43.953798 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:43.953972 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:43.957456 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:43.957462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:43.957466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:43.957497 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:43.960345 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.961083 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.961169 (+ 86us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.961173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.961326 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:43.961767 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:43.961772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:43.961775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:43.962129 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:43.962357 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:43.962773 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 22:43:43.963084 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":80}
I20260807 22:43:43.969300 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Bootstrap starting.
I20260807 22:43:43.973425 27835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:43.974786 27835 log.cc:824] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:43.978277 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: No bootstrap required, opened a new log
I20260807 22:43:43.986855 27835 raft_consensus.cc:348] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } }
I20260807 22:43:43.987305 27835 raft_consensus.cc:374] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:43.987430 27835 raft_consensus.cc:729] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de12ce6fb83b4c07ad867beb0fb35bb0, State: Initialized, Role: FOLLOWER
I20260807 22:43:43.988237 27835 consensus_queue.cc:260] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } }
I20260807 22:43:43.989346 27843 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } } }
I20260807 22:43:43.989661 27843 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:43.990549 27835 sys_catalog.cc:564] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:43:44.000952 27854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:43:44.001137 27854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:43:44.006284 27759 master_runner.cc:428] Detected that this master de12ce6fb83b4c07ad867beb0fb35bb0 is joining an existing cluster
I20260807 22:43:44.006428 27759 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.10.189:43165
I20260807 22:43:44.030615 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.012072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.20.10.190:43009)
0807 22:43:44.012275 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.012280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.015082 (+ 2802us) server_negotiation.cc:407] Connection header received
0807 22:43:44.015380 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.015387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.015501 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.015624 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.016740 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.017475 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.019322 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.019979 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.020140 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.023015 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.023022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.023026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.023070 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.026735 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.027671 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.027678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.027682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.027907 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.028424 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.028429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.028433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.028855 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.029023 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.029795 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.030146 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:43:44.032310 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.011546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38544)
0807 22:43:44.014947 (+ 3401us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.014972 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.015218 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.015667 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.015672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.016252 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.016557 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.016567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.017636 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.017648 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.019098 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.019116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.020435 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.020442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.020644 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.021487 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.021512 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.022744 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.026923 (+ 4179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.026929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.026933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.027502 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.028082 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.028087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.028090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.028284 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.029121 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.029128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.029420 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.031335 (+ 1915us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.031749 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2984,"thread_start_us":7477,"threads_started":1}
WARNING: no policy specified for oryx/127.20.10.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.188@KRBTEST.COM" created.
I20260807 22:43:44.052618 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.011497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.20.10.189:43165)
0807 22:43:44.011715 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.011724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.019759 (+ 8035us) server_negotiation.cc:407] Connection header received
0807 22:43:44.020141 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.020146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.020217 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.020282 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.022417 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.025366 (+ 2949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.028651 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.029407 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.029512 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.032269 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.032277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.032283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.032323 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.036880 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.037994 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.037999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.038002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.038253 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.038898 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.038904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.038906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.039356 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.039549 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.051477 (+ 11928us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.051934 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:43:44.054358 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.012068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39928)
0807 22:43:44.019655 (+ 7587us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.019677 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.020014 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.020490 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.020494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.021830 (+ 1336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.022242 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.022257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.025644 (+ 3387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.025650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.027365 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.027381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.029845 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.029853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.030020 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.030780 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.030809 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.032060 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.037071 (+ 5011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.037077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.037082 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.037780 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.038425 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.038432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.038436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.038699 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.039612 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.039620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.050921 (+ 11301us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.053123 (+ 2202us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.053708 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7422,"thread_start_us":2282,"threads_started":1}
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142621, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.190@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
I20260807 22:43:44.073061 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.041577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.20.10.189:43165)
0807 22:43:44.053055 (+ 11478us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.053064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.054864 (+ 1800us) server_negotiation.cc:407] Connection header received
0807 22:43:44.055454 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.055461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.055593 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.055689 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.057785 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.059301 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.060800 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.061410 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.061542 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.066226 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.066232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.066236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.066275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.069470 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.070532 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.070540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.070543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.070767 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.071293 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.071299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.071302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.071667 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.071823 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.072322 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.072590 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11352,"thread_start_us":12297,"threads_started":1}
I20260807 22:43:44.073139 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.041519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39938)
0807 22:43:44.054637 (+ 13118us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.054758 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.055053 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.056055 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.056065 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.057121 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.057491 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.057509 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.059504 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.059511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.060582 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.060599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.061820 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.061830 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.062094 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.063048 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.063074 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.065965 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.069659 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.069678 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.069687 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.070393 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.070904 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.070910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.070913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.071173 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.071815 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.071824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.072015 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.072227 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.072688 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12752,"thread_start_us":12581,"threads_started":1}
I20260807 22:43:44.075843 27259 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.10.189:43165: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
member_type: NON_VOTER
last_known_addr {
host: "127.20.10.189"
port: 43165
}
attrs {
promote: true
}
}
dest_uuid: "6f36bc52d5b1432283883efed09e4aea"
cas_config_opid_index: -1
I20260807 22:43:44.076905 27259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } }
I20260807 22:43:44.080464 27870 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } } }
I20260807 22:43:44.081912 27807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:44.086592 27807 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 22:43:44.086638 27870 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.10.189:43165
I20260807 22:43:44.087858 27759 master.cc:537] Master@127.20.10.189:43165 shutting down...
I20260807 22:43:44.088745 27871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:44.089649 27869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } } }
I20260807 22:43:44.090058 27869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.188.keytab.
Entry for principal oryx/127.20.10.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.188.keytab.
W20260807 22:43:44.093508 27244 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 22:43:44.094601 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:42771
--webserver_interface=127.20.10.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.188.keytab
--principal=oryx/127.20.10.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:44.096235 27244 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea -> Peer de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189:43165): Couldn't send request to peer de12ce6fb83b4c07ad867beb0fb35bb0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:44.104810 27759 raft_consensus.cc:2229] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:44.105352 27759 raft_consensus.cc:2258] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:44.105588 27759 tablet_replica.cc:330] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: stopping tablet replica
I20260807 22:43:44.114508 27759 master.cc:552] Master@127.20.10.189:43165 shutdown complete.
I20260807 22:43:44.114754 27759 master_runner.cc:305] Clearing existing system tablet
I20260807 22:43:44.115335 27759 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:44.115648 27759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:44.119421 27759 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 22:43:44.121501 27876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:44.121986 27759 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:44.122313 27759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 22:43:44.122429 27759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "c998994a2359f865641bec3e84a2c651"
server_key_iv: "78a5ee362df99316bfc178719c44d02f"
server_key_version: "encryptionkey@0"
I20260807 22:43:44.128927 27759 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:44.137383 27759 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:43:44.137631 27759 log.cc:1196] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 22:43:44.138185 27759 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Deleting consensus metadata
I20260807 22:43:44.138499 27759 master_runner.cc:315] Copying system tablet from 127.20.10.190:43009
I20260807 22:43:44.143900 27759 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: tablet copy: Beginning tablet copy session from remote peer at address 127.20.10.190:43009
I20260807 22:43:44.166847 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.144965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38560)
0807 22:43:44.151105 (+ 6140us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.151136 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.151332 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.151815 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.151820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.152411 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.152718 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.152731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.154265 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.154271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.155952 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.155971 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.157039 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.157046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.157240 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.158002 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.158029 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.159206 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.163408 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.163415 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.163419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.164010 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.164547 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.164553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.164558 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.164761 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.165573 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.165579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.165791 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.165958 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.166311 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5945,"thread_start_us":5818,"threads_started":1}
I20260807 22:43:44.167325 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.145031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.20.10.190:43009)
0807 22:43:44.145261 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.145268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.151295 (+ 6027us) server_negotiation.cc:407] Connection header received
0807 22:43:44.151505 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.151511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.151607 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.151744 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.152943 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.154083 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.156150 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.156852 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.156999 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.159467 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.159474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.159478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.159522 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.163222 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.164193 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.164200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.164207 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.164405 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.164914 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.164921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.164927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.165422 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.165612 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.166150 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.166453 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 22:43:44.168286 27282 tablet_copy_service.cc:140] P 6f36bc52d5b1432283883efed09e4aea: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer de12ce6fb83b4c07ad867beb0fb35bb0 ({username='oryx', principal='oryx/127.20.10.189@KRBTEST.COM'} at 127.0.0.1:38560)
I20260807 22:43:44.168637 27282 tablet_copy_service.cc:161] P 6f36bc52d5b1432283883efed09e4aea: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer de12ce6fb83b4c07ad867beb0fb35bb0 at {username='oryx', principal='oryx/127.20.10.189@KRBTEST.COM'} at 127.0.0.1:38560: session id = de12ce6fb83b4c07ad867beb0fb35bb0-00000000000000000000000000000000
I20260807 22:43:44.173549 27282 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:43:44.176092 27759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:43:44.183678 27759 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: tablet copy: Starting download of 0 data blocks...
I20260807 22:43:44.184049 27759 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: tablet copy: Starting download of 1 WAL segments...
I20260807 22:43:44.187451 27759 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:43:44.192898 27282 tablet_copy_service.cc:342] P 6f36bc52d5b1432283883efed09e4aea: Request end of tablet copy session de12ce6fb83b4c07ad867beb0fb35bb0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.10.189@KRBTEST.COM'} at 127.0.0.1:38560
I20260807 22:43:44.193224 27282 tablet_copy_service.cc:433] P 6f36bc52d5b1432283883efed09e4aea: ending tablet copy session de12ce6fb83b4c07ad867beb0fb35bb0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer de12ce6fb83b4c07ad867beb0fb35bb0
I20260807 22:43:44.207914 27759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:44.211053 27883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:44.211676 27884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:44.219306 27759 server_base.cc:1034] running on GCE node
W20260807 22:43:44.219396 27887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:44.220249 27759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:44.221037 27759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:44.222270 27759 hybrid_clock.cc:648] HybridClock initialized: now 1786142624222212 us; error 35 us; skew 500 ppm
I20260807 22:43:44.222566 27759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142624, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:44.225330 27759 init.cc:377] Logged in from keytab as oryx/127.20.10.189@KRBTEST.COM (short username oryx)
I20260807 22:43:44.227057 27759 webserver.cc:460] Webserver started at http://127.20.10.189:46135/ using document root <none> and password file <none>
I20260807 22:43:44.227567 27759 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:44.227732 27759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:44.231779 27759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:44.234421 27894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:44.234938 27759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:44.235159 27759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "c998994a2359f865641bec3e84a2c651"
server_key_iv: "78a5ee362df99316bfc178719c44d02f"
server_key_version: "encryptionkey@0"
I20260807 22:43:44.235416 27759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:44.265511 27759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:44.270453 27759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:44.316402 27753 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
I20260807 22:43:44.320299 27759 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.189:43165
I20260807 22:43:44.320393 27954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.189:43165 every 8 connection(s)
I20260807 22:43:44.321532 27759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
I20260807 22:43:44.362994 27955 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:43:44.376307 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Bootstrap starting.
I20260807 22:43:44.395538 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:43:44.396262 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: Bootstrap complete.
I20260807 22:43:44.397318 27955 raft_consensus.cc:348] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } }
I20260807 22:43:44.397614 27955 raft_consensus.cc:729] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: de12ce6fb83b4c07ad867beb0fb35bb0, State: Initialized, Role: LEARNER
I20260807 22:43:44.397890 27955 consensus_queue.cc:260] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } }
I20260807 22:43:44.399251 27958 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } } }
I20260807 22:43:44.399781 27958 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:44.399946 27955 sys_catalog.cc:564] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:44.419044 27969 catalog_manager.cc:1259] Loaded cluster ID: ace5f31a27fa4c9cb79d54f2f3af9199
I20260807 22:43:44.419196 27969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: loading cluster ID for follower catalog manager: success
I20260807 22:43:44.422076 27969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: acquiring CA information for follower catalog manager: success
I20260807 22:43:44.424489 27969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 22:43:44.579582 27874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:44.580430 27874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:44.580721 27874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:44.601969 27874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:44.602355 27874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:44.602579 27874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:44.602849 27874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:44.602945 27874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:44.628320 27874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.188.keytab
--principal=oryx/127.20.10.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.10.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:44.631110 27874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:44.633965 27874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:44.660334 27974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:44.661113 27976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:44.661760 27979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:44.665480 27874 server_base.cc:1034] running on GCE node
I20260807 22:43:44.667244 27874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:44.677943 27874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:44.680166 27874 hybrid_clock.cc:648] HybridClock initialized: now 1786142624679975 us; error 157 us; skew 500 ppm
I20260807 22:43:44.680778 27874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:43:44.681676 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.658557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39952)
0807 22:43:44.660125 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.660167 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.660426 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.662249 (+ 1823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.662258 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.662307 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:44.662604 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.662621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.665519 (+ 2898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.665526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.667973 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.668308 (+ 335us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.680025 (+ 11717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.680045 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.680412 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.680542 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.680691 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.680993 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":1592,"threads_started":1}
I20260807 22:43:44.684296 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.659726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.20.10.189:43165)
0807 22:43:44.661588 (+ 1862us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.661597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.661621 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:43:44.661814 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.661821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.661987 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.662151 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:44.662934 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.665242 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.668539 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.679214 (+ 10675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.679459 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.682484 (+ 3025us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.682752 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.683014 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1605,"thread_start_us":1288,"threads_started":1}
I20260807 22:43:44.686553 27924 raft_consensus.cc:1203] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142624, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:44.687927 27958 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f36bc52d5b1432283883efed09e4aea. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } } }
I20260807 22:43:44.688305 27958 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:44.689610 27874 init.cc:377] Logged in from keytab as oryx/127.20.10.188@KRBTEST.COM (short username oryx)
I20260807 22:43:44.694276 27874 webserver.cc:460] Webserver started at http://127.20.10.188:35335/ using document root <none> and password file <none>
I20260807 22:43:44.695971 27874 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:44.696348 27874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:44.696952 27874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:44.704492 27874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/instance:
uuid: "d231b5f21b2e409599df65ad0ca5d224"
format_stamp: "Formatted at 2026-08-07 22:43:44 on dist-test-slave-csg5"
server_key: "79a68341087983f853a17056e4791d04"
server_key_iv: "a6b820254dac9c720f2ba985be19524a"
server_key_version: "encryptionkey@0"
I20260807 22:43:44.706274 27874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal/instance:
uuid: "d231b5f21b2e409599df65ad0ca5d224"
format_stamp: "Formatted at 2026-08-07 22:43:44 on dist-test-slave-csg5"
server_key: "79a68341087983f853a17056e4791d04"
server_key_iv: "a6b820254dac9c720f2ba985be19524a"
server_key_version: "encryptionkey@0"
I20260807 22:43:44.718941 27874 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.009s sys 0.001s
I20260807 22:43:44.727164 27989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:44.729465 27874 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260807 22:43:44.729811 27874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "d231b5f21b2e409599df65ad0ca5d224"
format_stamp: "Formatted at 2026-08-07 22:43:44 on dist-test-slave-csg5"
server_key: "79a68341087983f853a17056e4791d04"
server_key_iv: "a6b820254dac9c720f2ba985be19524a"
server_key_version: "encryptionkey@0"
I20260807 22:43:44.730247 27874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:44.756094 27874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:44.763983 27874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:44.764530 27874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:44.818567 27874 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.188:42771
I20260807 22:43:44.818670 28046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.188:42771 every 8 connection(s)
I20260807 22:43:44.820716 27874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
I20260807 22:43:44.827127 28047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:44.829068 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 27874
I20260807 22:43:44.843801 28047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142624, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.188@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:44.871897 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.832178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.20.10.188:42771)
0807 22:43:44.834614 (+ 2436us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.834633 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.834752 (+ 119us) server_negotiation.cc:407] Connection header received
0807 22:43:44.838876 (+ 4124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.838911 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.839666 (+ 755us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.840393 (+ 727us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.841902 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.844717 (+ 2815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.846712 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.847654 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.847889 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.857716 (+ 9827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.857728 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.857742 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.857823 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.864181 (+ 6358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.865806 (+ 1625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.865814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.865823 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.866321 (+ 498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.867087 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.867092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.867094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.868083 (+ 989us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.868496 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.869251 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.869862 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1764,"thread_start_us":1238,"threads_started":1}
I20260807 22:43:44.881768 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.848565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38576)
0807 22:43:44.850801 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.850848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.851081 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.855847 (+ 4766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.855854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.857239 (+ 1385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.858258 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.858274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.860511 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.860518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.862998 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.863016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.864842 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.864856 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.865304 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.866631 (+ 1327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.866701 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.871316 (+ 4615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.877715 (+ 6399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.877726 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.877731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.878606 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.879282 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.879286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.879289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.879571 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.880479 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.880485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.880667 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.880885 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.881207 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1802,"thread_start_us":1415,"threads_started":1}
I20260807 22:43:44.881834 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.848538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.20.10.190:43009)
0807 22:43:44.855272 (+ 6734us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.855278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.855305 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:44.855376 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.855380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.855559 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.855713 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.858540 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.860065 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.863204 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.864338 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.864479 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.871757 (+ 7278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.871781 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.871787 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.871863 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.877542 (+ 5679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.878819 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.878825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.878827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.879117 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.879778 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.879782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.879784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.880267 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.880429 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.881046 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.881277 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6541,"thread_start_us":6394,"threads_started":1}
I20260807 22:43:44.883621 28047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142624, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.188@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
I20260807 22:43:44.906167 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.884441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39968)
0807 22:43:44.884772 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.884796 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.884960 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.885566 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.885570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.886320 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.886735 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.886750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.888375 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.888380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.890049 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.890065 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.891535 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.891541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.891837 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.892941 (+ 1104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.892963 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.896832 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.901785 (+ 4953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.901791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.901794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.903014 (+ 1220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.903853 (+ 839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.903858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.903861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.904243 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.905067 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.905071 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.905373 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.905506 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.905831 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 22:43:44.906838 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.884945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.20.10.189:43165)
0807 22:43:44.885189 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.885194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.885211 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:44.885262 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.885265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.885325 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.885407 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.886947 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.888093 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.890278 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.891262 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.891399 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.897147 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.897155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.897159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.897200 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.901584 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.903220 (+ 1636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.903226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.903228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.903687 (+ 459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.904421 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.904426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.904429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.904908 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.905053 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.905738 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.906050 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:43:44.908306 28047 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142624, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.188@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
I20260807 22:43:44.929392 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.909327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.20.10.188:42771)
0807 22:43:44.909568 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.909574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.909673 (+ 99us) server_negotiation.cc:407] Connection header received
0807 22:43:44.909878 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.909881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.909933 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.910031 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.911292 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.912278 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.914002 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.914698 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.914893 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.919841 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:44.919846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:44.919849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:44.919883 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:44.924778 (+ 4895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.925816 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.925824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.925827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.926045 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:44.926700 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:44.926712 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:44.926718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:44.927428 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:44.927666 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:44.928402 (+ 736us) server_negotiation.cc:299] Negotiation successful
0807 22:43:44.928790 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 22:43:44.929509 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.909327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.0.0.1:40156)
0807 22:43:44.909568 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:44.909590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:44.909750 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:44.910047 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:44.910051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:44.910809 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:44.911141 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.911152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.912413 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.912422 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.913833 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:44.913853 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.914956 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.914962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:44.915389 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:44.916302 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:44.916320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:44.919557 (+ 3237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:44.925000 (+ 5443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.925007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.925011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.925672 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.926191 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:44.926195 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:44.926199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:44.926459 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:44.927590 (+ 1131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:44.927595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:44.927905 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 22:43:44.928134 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 22:43:44.928885 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":258}
I20260807 22:43:44.938153 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Bootstrap starting.
I20260807 22:43:44.943831 28047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:44.946209 28047 log.cc:824] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:44.951257 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: No bootstrap required, opened a new log
I20260807 22:43:44.964905 28047 raft_consensus.cc:348] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } }
I20260807 22:43:44.965528 28047 raft_consensus.cc:374] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:44.965795 28047 raft_consensus.cc:729] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d231b5f21b2e409599df65ad0ca5d224, State: Initialized, Role: FOLLOWER
I20260807 22:43:44.966562 28047 consensus_queue.cc:260] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } }
I20260807 22:43:44.967954 28055 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } } }
I20260807 22:43:44.968420 28055 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:44.969312 28047 sys_catalog.cc:564] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:44.981184 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27327
W20260807 22:43:44.982149 28068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:43:44.982338 28068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:43:44.992918 27874 master_runner.cc:428] Detected that this master d231b5f21b2e409599df65ad0ca5d224 is joining an existing cluster
I20260807 22:43:44.993259 27874 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.10.188:42771
Aug 07 22:43:44 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.129@KRBTEST.COM
I20260807 22:43:45.016965 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:44.982774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.20.10.129:41251)
0807 22:43:44.983620 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:44.983630 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:44.983664 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:43:44.983739 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:44.983748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:44.984075 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:44.986189 (+ 2114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:44.986497 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.988286 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.992425 (+ 4139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:44.993324 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:44.993595 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.002421 (+ 8826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.002444 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.002490 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.002602 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.009539 (+ 6937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.011229 (+ 1690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.011240 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.011242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.011523 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.012404 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.012422 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.012426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.013327 (+ 901us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.013888 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.014959 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.015451 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":367,"threads_started":1}
W20260807 22:43:45.019641 27996 tablet.cc:2369] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:45.021447 27461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:45.063084 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.009212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.20.10.190:43009)
0807 22:43:45.009443 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.009450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.013857 (+ 4407us) server_negotiation.cc:407] Connection header received
0807 22:43:45.014337 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.014344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.014449 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.014538 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:45.018935 (+ 4397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.020559 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.025256 (+ 4697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.026466 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.026730 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.037697 (+ 10967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.037704 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.037708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.037789 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.044876 (+ 7087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.051371 (+ 6495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.051378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.051382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.051677 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.054774 (+ 3097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.054780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.054783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.055439 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.055621 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.061824 (+ 6203us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.062219 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:43:45.070340 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.010608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38592)
0807 22:43:45.013720 (+ 3112us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.013746 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.014063 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.014859 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.014864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.017187 (+ 2323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:45.018492 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.018520 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.020970 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.020974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.024486 (+ 3512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.024512 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.027785 (+ 3273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.027805 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.028285 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.030887 (+ 2602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:45.030927 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:45.037290 (+ 6363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:45.045107 (+ 7817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.045115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.045120 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.051016 (+ 5896us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.053553 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.053562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.053570 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.054600 (+ 1030us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.060734 (+ 6134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:45.060742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:45.061188 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.061372 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.061855 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2827,"thread_start_us":16606,"threads_started":1}
I20260807 22:43:45.070523 27980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: NON_VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: true } } }
I20260807 22:43:45.073146 27980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:45.076280 28082 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: attempting to promote NON_VOTER de12ce6fb83b4c07ad867beb0fb35bb0 to VOTER
I20260807 22:43:45.078078 28082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } }
I20260807 22:43:45.085165 27924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 LEARNER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:43:45.086089 28083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 22:43:45.094763 28083 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } }
I20260807 22:43:45.100986 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } }
I20260807 22:43:45.101434 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:45.101945 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } }
I20260807 22:43:45.102339 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:45.109898 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.014942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.0.0.1:40160)
0807 22:43:45.035969 (+ 21027us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.036000 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.037703 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.038463 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.038470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.046362 (+ 7892us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:45.054394 (+ 8032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.054421 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.058635 (+ 4214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.058641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.063324 (+ 4683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.063342 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.065468 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.065477 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.065814 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.067058 (+ 1244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:45.067198 (+ 140us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:45.068995 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:45.105450 (+ 36455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.105456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.105461 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.106161 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.106773 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.106778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.106782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.106999 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.108309 (+ 1310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:45.108314 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:45.108562 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.108723 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.109067 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19825,"mutex_wait_us":1,"thread_start_us":19404,"threads_started":1}
I20260807 22:43:45.111351 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.014439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.20.10.188:42771)
0807 22:43:45.014793 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.014799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.036134 (+ 21335us) server_negotiation.cc:407] Connection header received
0807 22:43:45.037921 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.037930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.038041 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.038129 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:45.056532 (+ 18403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.058227 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.063486 (+ 5259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.064734 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.064838 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.070008 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.070025 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.070040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.070101 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.105218 (+ 35117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.106304 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.106309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.106313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.106602 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.107318 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.107324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.107328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.108090 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.108257 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.110458 (+ 2201us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.110815 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 22:43:45.118537 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.009357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39980 (local address 127.20.10.189:43165)
0807 22:43:45.009603 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.009612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.028878 (+ 19266us) server_negotiation.cc:407] Connection header received
0807 22:43:45.029373 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.029381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.029490 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.029582 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:45.043705 (+ 14123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.045439 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.054944 (+ 9505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.056091 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.056208 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.096542 (+ 40334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.096554 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.096563 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.096631 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.114175 (+ 17544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.115289 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.115296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.115300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.115594 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.116159 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.116163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.116167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.116586 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.117079 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.117512 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.117891 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 22:43:45.118845 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.011369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:39980)
0807 22:43:45.028733 (+ 17364us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.028759 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.029091 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.030001 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.030008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.035444 (+ 5436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:45.041171 (+ 5727us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x61200002d828. stack: 00007fb522df066e 00007fb522df0430 00007fb521e775ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fb522de60fd 00007fb522de6977 00007fb537194a7d 00007fb537194a0e 00007fb523f1f7b7 00007fb5297f403a 00007fb52987bdca 00007fb529879f68 00007fb5298ad82f 00007fb5298ad58d
0807 22:43:45.042202 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.042240 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.048389 (+ 6149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.048401 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.054773 (+ 6372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.054784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.066314 (+ 11530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.066320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.066622 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.092431 (+ 25809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:45.092485 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:45.096202 (+ 3717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:45.114379 (+ 18177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.114384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.114390 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.115115 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.115757 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.115762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.115767 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.116002 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.116784 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:45.116789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:45.117155 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.117363 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.117895 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17089,"mutex_wait_us":415,"spinlock_wait_cycles":5041536,"thread_start_us":1590,"threads_started":1}
I20260807 22:43:45.117439 27924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } }
I20260807 22:43:45.126078 27958 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } }
I20260807 22:43:45.126550 27958 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 22:43:45 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142621, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.190@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
I20260807 22:43:45.149386 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.109205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.0.0.1:40172)
0807 22:43:45.109744 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.109768 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.109897 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.114616 (+ 4719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.114623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.115830 (+ 1207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:45.116355 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.116382 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.126286 (+ 9904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.126295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.129498 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.129525 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.131794 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.131801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.132261 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.133360 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:45.133385 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:45.138234 (+ 4849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:45.144879 (+ 6645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.144886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.144890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.145965 (+ 1075us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.146625 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.146632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.146635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.146926 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.147803 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:45.147809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:45.148119 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.148321 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.148751 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349}
I20260807 22:43:45.149401 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.110156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.20.10.188:42771)
0807 22:43:45.111998 (+ 1842us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.112004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.112022 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:43:45.112178 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.112183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.112276 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.112372 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:45.121126 (+ 8754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.123443 (+ 2317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.129842 (+ 6399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.131428 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.131612 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.138577 (+ 6965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.138585 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.138591 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.138639 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.144629 (+ 5990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.146191 (+ 1562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.146198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.146202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.146443 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.147078 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.147083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.147087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.147642 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.147787 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.148462 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.148729 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1723,"thread_start_us":13433,"threads_started":1}
I20260807 22:43:45.151995 27259 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.10.188:42771: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224"
member_type: NON_VOTER
last_known_addr {
host: "127.20.10.188"
port: 42771
}
attrs {
promote: true
}
}
dest_uuid: "6f36bc52d5b1432283883efed09e4aea"
cas_config_opid_index: 6
I20260807 22:43:45.153223 27259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } }
I20260807 22:43:45.156796 27924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:43:45.156736 28018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:45.158041 27980 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 22:43:45.164232 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27327
I20260807 22:43:45.165305 27980 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.165799 28018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:43:45.171633 27980 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.10.188:42771
I20260807 22:43:45.171578 28083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:45.172549 27874 master.cc:537] Master@127.20.10.188:42771 shutting down...
I20260807 22:43:45.174836 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.175192 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:45.175585 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.175817 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:45.175724 27924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.180784 27958 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.181131 27958 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:45.192348 27874 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:45.193128 27874 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:45.193447 27874 tablet_replica.cc:330] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: stopping tablet replica
W20260807 22:43:45.204942 27244 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea -> Peer d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188:42771): Couldn't send request to peer d231b5f21b2e409599df65ad0ca5d224. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:45.208201 27874 master.cc:552] Master@127.20.10.188:42771 shutdown complete.
I20260807 22:43:45.208326 27874 master_runner.cc:305] Clearing existing system tablet
I20260807 22:43:45.208823 27874 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:45.208966 27874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260807 22:43:45.209198 27243 connection.cc:535] server connection from 127.20.10.129:49923 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:43:45.210803 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:41251
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:43009
--webserver_port=43871
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:45.212214 27874 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.005s sys 0.000s
I20260807 22:43:45.214363 28089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:45.214902 27874 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:45.215206 27874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 22:43:45.215329 27874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "d231b5f21b2e409599df65ad0ca5d224"
format_stamp: "Formatted at 2026-08-07 22:43:44 on dist-test-slave-csg5"
server_key: "79a68341087983f853a17056e4791d04"
server_key_iv: "a6b820254dac9c720f2ba985be19524a"
server_key_version: "encryptionkey@0"
I20260807 22:43:45.220719 27874 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:45.230057 27874 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:43:45.230291 27874 log.cc:1196] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 22:43:45.230883 27874 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Deleting consensus metadata
I20260807 22:43:45.231391 27874 master_runner.cc:315] Copying system tablet from 127.20.10.190:43009
I20260807 22:43:45.235110 27874 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: tablet copy: Beginning tablet copy session from remote peer at address 127.20.10.190:43009
I20260807 22:43:45.255316 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.236898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.20.10.190:43009)
0807 22:43:45.237244 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.237251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.237626 (+ 375us) server_negotiation.cc:407] Connection header received
0807 22:43:45.237920 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.237926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.238044 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.238160 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:45.239593 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.241036 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.243768 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.244517 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.244728 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.247936 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:45.247945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:45.247950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:45.247998 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:45.251907 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.252840 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.252846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.252850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.253028 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:45.253501 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:45.253507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:45.253510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:45.253939 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:45.254095 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.254643 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.254938 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260807 22:43:45.255322 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.236112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:38604)
0807 22:43:45.237474 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.237505 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.237698 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.238252 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.238260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.239029 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:45.239376 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.239394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.241279 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.241288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.243425 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.243449 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.245022 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.245029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.245478 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.246285 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:45.246319 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:45.247649 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:45.252104 (+ 4455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.252110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.252113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.252679 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.253163 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:45.253168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:45.253173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:45.253370 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:45.254083 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:45.254088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:45.254298 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.254456 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.254831 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":1133,"threads_started":1}
I20260807 22:43:45.256490 27282 tablet_copy_service.cc:140] P 6f36bc52d5b1432283883efed09e4aea: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d231b5f21b2e409599df65ad0ca5d224 ({username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:38604)
I20260807 22:43:45.256703 27282 tablet_copy_service.cc:161] P 6f36bc52d5b1432283883efed09e4aea: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d231b5f21b2e409599df65ad0ca5d224 at {username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:38604: session id = d231b5f21b2e409599df65ad0ca5d224-00000000000000000000000000000000
I20260807 22:43:45.260596 27282 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:43:45.262832 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:43:45.270426 27874 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: tablet copy: Starting download of 0 data blocks...
I20260807 22:43:45.270818 27874 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: tablet copy: Starting download of 1 WAL segments...
I20260807 22:43:45.273847 27874 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:43:45.281193 27282 tablet_copy_service.cc:342] P 6f36bc52d5b1432283883efed09e4aea: Request end of tablet copy session d231b5f21b2e409599df65ad0ca5d224-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:38604
I20260807 22:43:45.281383 27282 tablet_copy_service.cc:433] P 6f36bc52d5b1432283883efed09e4aea: ending tablet copy session d231b5f21b2e409599df65ad0ca5d224-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d231b5f21b2e409599df65ad0ca5d224
I20260807 22:43:45.293627 27874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:45.296977 28097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:45.297248 28098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:45.305455 28100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:45.306041 27874 server_base.cc:1034] running on GCE node
I20260807 22:43:45.306427 27874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:45.307189 27874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:45.308398 27874 hybrid_clock.cc:648] HybridClock initialized: now 1786142625308352 us; error 40 us; skew 500 ppm
I20260807 22:43:45.308676 27874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:45 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142625, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:45.310863 27874 init.cc:377] Logged in from keytab as oryx/127.20.10.188@KRBTEST.COM (short username oryx)
I20260807 22:43:45.344633 27874 webserver.cc:460] Webserver started at http://127.20.10.188:46317/ using document root <none> and password file <none>
I20260807 22:43:45.345099 27874 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:45.345240 27874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:45.348989 27874 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:45.351176 28110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:45.351662 27874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:45.351794 27874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "d231b5f21b2e409599df65ad0ca5d224"
format_stamp: "Formatted at 2026-08-07 22:43:44 on dist-test-slave-csg5"
server_key: "79a68341087983f853a17056e4791d04"
server_key_iv: "a6b820254dac9c720f2ba985be19524a"
server_key_version: "encryptionkey@0"
I20260807 22:43:45.352007 27874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:45.367720 27874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:45.372118 27874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:45.405534 27874 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.188:42771
I20260807 22:43:45.405725 28166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.188:42771 every 8 connection(s)
I20260807 22:43:45.406751 27874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
I20260807 22:43:45.412612 28167 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:43:45.417918 28167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Bootstrap starting.
I20260807 22:43:45.437772 28167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:43:45.438351 28167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Bootstrap complete.
I20260807 22:43:45.439576 28167 raft_consensus.cc:348] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } }
I20260807 22:43:45.439919 28167 raft_consensus.cc:729] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d231b5f21b2e409599df65ad0ca5d224, State: Initialized, Role: LEARNER
I20260807 22:43:45.440178 28167 consensus_queue.cc:260] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } }
I20260807 22:43:45.447590 28167 sys_catalog.cc:564] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:45.447307 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.448173 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:45.455796 28182 catalog_manager.cc:1259] Loaded cluster ID: ace5f31a27fa4c9cb79d54f2f3af9199
I20260807 22:43:45.455968 28182 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: loading cluster ID for follower catalog manager: success
I20260807 22:43:45.459594 28182 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: acquiring CA information for follower catalog manager: success
I20260807 22:43:45.462412 28182 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:43:45.592502 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.578818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.20.10.188:42771)
0807 22:43:45.579900 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.579905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.579929 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:43:45.580006 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.580011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.580113 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.580382 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:45.581337 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.583561 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.588065 (+ 4504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.589854 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.590078 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.590476 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:45.591454 (+ 978us) server_negotiation.cc:299] Negotiation successful
0807 22:43:45.591661 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":530,"threads_started":1}
I20260807 22:43:45.593225 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.577788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.0.0.1:40180)
0807 22:43:45.578085 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.578109 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.578431 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.580614 (+ 2183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.580622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:45.580657 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:45.580870 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.580888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.583884 (+ 2996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.583890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.587673 (+ 3783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:45.587708 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:45.590310 (+ 2602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:45.590322 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:45.590912 (+ 590us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:45.591023 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 22:43:45.591384 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 22:43:45.591772 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 22:43:45.596239 28140 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260807 22:43:45.597697 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f36bc52d5b1432283883efed09e4aea. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:45.598260 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: LEARNER
W20260807 22:43:45.736497 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:45.737109 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:45.737403 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:45.749626 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:45.749889 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:45.750234 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:45.766717 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:41251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=43871
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:45.768502 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:45.770205 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:45.794632 28188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:45.795181 28191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:45.795753 28189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:45.797711 28090 server_base.cc:1034] running on GCE node
I20260807 22:43:45.798832 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:45.800321 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:45.801678 28090 hybrid_clock.cc:648] HybridClock initialized: now 1786142625801579 us; error 70 us; skew 500 ppm
I20260807 22:43:45.802210 28090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:45 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142625, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:45.808645 28090 init.cc:377] Logged in from keytab as oryx/127.20.10.129@KRBTEST.COM (short username oryx)
I20260807 22:43:45.811311 28090 webserver.cc:460] Webserver started at http://127.20.10.129:43871/ using document root <none> and password file <none>
I20260807 22:43:45.812387 28090 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:45.812657 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:45.821314 28090 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 22:43:45.826634 28200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:45.828812 28090 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 22:43:45.829205 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "81ee3502329748c1af23a06553254d6e"
format_stamp: "Formatted at 2026-08-07 22:43:41 on dist-test-slave-csg5"
server_key: "7a3726d43e5fb46af62f798a7c614ab4"
server_key_iv: "dc35cc8de84a4f9c2568d3668f9b454f"
server_key_version: "encryptionkey@0"
I20260807 22:43:45.831075 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:45.850395 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:45.858398 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:45.859018 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:45.860720 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:45.863400 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:45.863590 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:45.863821 28090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:45.863971 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:45.973506 28090 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41251
I20260807 22:43:45.973820 28317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41251 every 8 connection(s)
I20260807 22:43:45.975788 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:45.983600 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 28090
I20260807 22:43:45.984074 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27474
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142625, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142625, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142625, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.130@KRBTEST.COM
I20260807 22:43:46.037602 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.981031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54767 (local address 127.20.10.188:42771)
0807 22:43:45.981300 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.981306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.992474 (+ 11168us) server_negotiation.cc:407] Connection header received
0807 22:43:45.999507 (+ 7033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.999513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.999600 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.999711 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.005628 (+ 5917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.007551 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.012214 (+ 4663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.013864 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.013994 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.024871 (+ 10877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.024882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.024890 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.024980 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.031542 (+ 6562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.033351 (+ 1809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.033359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.033363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.033692 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.034532 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.034538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.034541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.035038 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.035259 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.036658 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.036988 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:43:46.040460 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.982072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:49953)
0807 22:43:45.995794 (+ 13722us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.995878 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.998380 (+ 2502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:45.999462 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:45.999477 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.001407 (+ 1930us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.003717 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.003757 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.005341 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.005350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.009395 (+ 4045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.009416 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.010956 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.010974 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.011607 (+ 633us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.013157 (+ 1550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.013193 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.020865 (+ 7672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.032694 (+ 11829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.032702 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.032706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.034297 (+ 1591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.035483 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.035487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.035490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.035886 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.038463 (+ 2577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.038467 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.038619 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.038871 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.039219 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13445,"mutex_wait_us":3650,"spinlock_wait_cycles":28800,"thread_start_us":13546,"threads_started":1}
I20260807 22:43:46.040850 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.993583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.20.10.129:46835)
0807 22:43:45.997691 (+ 4108us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.997725 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.998872 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:46.007869 (+ 8997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:46.007874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.009715 (+ 1841us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.011522 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.011541 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.013910 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.013917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.017020 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.017035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.018829 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.018839 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.019253 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.020294 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.020321 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.026113 (+ 5792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.033539 (+ 7426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.033557 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.033566 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.034854 (+ 1288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.035474 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.035482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.035488 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.035872 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.036800 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.036806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.037134 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.037295 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.038003 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3874,"mutex_wait_us":30,"spinlock_wait_cycles":5504,"thread_start_us":4318,"threads_started":1}
I20260807 22:43:46.041028 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.980836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:49953 (local address 127.20.10.190:43009)
0807 22:43:45.982385 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.982395 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:45.996137 (+ 13742us) server_negotiation.cc:407] Connection header received
0807 22:43:45.998801 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:45.998807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:45.998945 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:45.999068 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.004000 (+ 4932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.005150 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.009635 (+ 4485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.010390 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.010569 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.021635 (+ 11066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.021647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.021655 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.021715 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.032099 (+ 10384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.034511 (+ 2412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.034522 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.034526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.034835 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.036058 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.036065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.036068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.036687 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.036875 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.039981 (+ 3106us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.040335 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1361,"thread_start_us":1200,"threads_started":1}
I20260807 22:43:46.041563 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.982072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.20.10.129:54767)
0807 22:43:45.992236 (+ 10164us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:45.992298 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:45.998380 (+ 6082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:46.000045 (+ 1665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:46.000050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.001407 (+ 1357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.003717 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.003757 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.007842 (+ 4085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.007849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.011955 (+ 4106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.011972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.014534 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.014546 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.014946 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.016114 (+ 1168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.016140 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.024445 (+ 8305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.031974 (+ 7529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.032001 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.032028 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.033124 (+ 1096us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.033907 (+ 783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.033911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.033915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.034392 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.035410 (+ 1018us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.035421 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.036077 (+ 656us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.037874 (+ 1797us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.038237 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9679,"mutex_wait_us":123,"spinlock_wait_cycles":21760,"thread_start_us":9101,"threads_started":1}
I20260807 22:43:46.042156 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.993385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:46835 (local address 127.20.10.189:43165)
0807 22:43:46.006805 (+ 13420us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:46.006812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.006860 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:43:46.006930 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.006935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.007114 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.007260 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.011780 (+ 4520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.013647 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.017212 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.018464 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.018644 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.026645 (+ 8001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.026684 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.026692 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.026751 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.033345 (+ 6594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.035013 (+ 1668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.035019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.035023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.035286 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.036077 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.036082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.036085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.036657 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.038802 (+ 2145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.038996 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.041453 (+ 2457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13222,"thread_start_us":14094,"threads_started":1}
I20260807 22:43:46.042688 28319 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:46.043252 28318 heartbeater.cc:347] Connected to a master server at 127.20.10.188:42771
I20260807 22:43:46.043279 28319 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.043469 28318 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.044346 28320 heartbeater.cc:347] Connected to a master server at 127.20.10.189:43165
I20260807 22:43:46.044541 28320 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.045323 28319 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
I20260807 22:43:46.045533 28318 heartbeater.cc:511] Master 127.20.10.188:42771 requested a full tablet report, sending...
I20260807 22:43:46.046046 28320 heartbeater.cc:511] Master 127.20.10.189:43165 requested a full tablet report, sending...
I20260807 22:43:46.048208 27259 ts_manager.cc:194] Re-registered known tserver with Master: 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251)
I20260807 22:43:46.048715 28127 ts_manager.cc:194] Registered new tserver with Master: 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251)
I20260807 22:43:46.049445 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:49953
I20260807 22:43:46.051082 27914 ts_manager.cc:194] Registered new tserver with Master: 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251)
I20260807 22:43:46.057816 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:45.985720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.20.10.130:38223)
0807 22:43:45.990513 (+ 4793us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:45.990530 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.002126 (+ 11596us) server_negotiation.cc:407] Connection header received
0807 22:43:46.002510 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.002524 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.003130 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.003316 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.008267 (+ 4951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.011158 (+ 2891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.014196 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.015783 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.015964 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.035710 (+ 19746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.035799 (+ 89us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.035824 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.035923 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.052340 (+ 16417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.053981 (+ 1641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.053988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.053991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.054251 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.054975 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.054979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.054981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.055505 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.055860 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.056681 (+ 821us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.056993 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4406,"thread_start_us":4016,"threads_started":1}
I20260807 22:43:46.059648 27603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:46.151985 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: NON_VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: true } } }
I20260807 22:43:46.152501 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:46.154469 28332 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: attempting to promote NON_VOTER d231b5f21b2e409599df65ad0ca5d224 to VOTER
I20260807 22:43:46.155493 28332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:46.156608 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27474
I20260807 22:43:46.159025 28140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 LEARNER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 22:43:46.159258 27924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Refusing update from remote peer 6f36bc52d5b1432283883efed09e4aea: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 22:43:46.160034 28083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 22:43:46.160509 28332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 22:43:46.163501 28332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.164415 27924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.167768 28332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.168025 28332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:46.168267 28332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.168511 28332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:46.171802 28335 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.172448 28335 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:46.173404 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.173694 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:46.174175 28140 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.177778 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f36bc52d5b1432283883efed09e4aea" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:46.178167 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:46.187126 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:38223
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:43009
--webserver_port=39117
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:46.590342 28337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:46.590750 28337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:46.590921 28337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:46.598807 28337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:46.598974 28337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:46.599169 28337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:46.609833 28337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:38223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=39117
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:46.611497 28337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:46.613162 28337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:46.632138 28344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:46.632090 28342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:46.632429 28337 server_base.cc:1034] running on GCE node
W20260807 22:43:46.632175 28346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:46.633617 28337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:46.634953 28337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:46.636456 28337 hybrid_clock.cc:648] HybridClock initialized: now 1786142626636158 us; error 128 us; skew 500 ppm
I20260807 22:43:46.636890 28337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142626, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:46.647710 28337 init.cc:377] Logged in from keytab as oryx/127.20.10.130@KRBTEST.COM (short username oryx)
I20260807 22:43:46.650174 28337 webserver.cc:460] Webserver started at http://127.20.10.130:39117/ using document root <none> and password file <none>
I20260807 22:43:46.651208 28337 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:46.651536 28337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:46.658048 28337 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:43:46.661861 28354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:46.663714 28337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 22:43:46.663960 28337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
format_stamp: "Formatted at 2026-08-07 22:43:42 on dist-test-slave-csg5"
server_key: "62de8a889e6f26072ed9c1f609fbb8cb"
server_key_iv: "284bc1fa83ae24b7491875a496924986"
server_key_version: "encryptionkey@0"
I20260807 22:43:46.665122 28337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:46.702463 28337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:46.708703 28337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:46.709038 28337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:46.710302 28337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:46.712158 28337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:46.712286 28337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:46.712376 28337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:46.712440 28337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:46.756552 28337 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:38223
I20260807 22:43:46.756654 28479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:38223 every 8 connection(s)
I20260807 22:43:46.758219 28337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:46.761866 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 28337
I20260807 22:43:46.762229 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27616
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142626, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142626, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.131@KRBTEST.COM
Aug 07 22:43:46 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142626, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
I20260807 22:43:46.847100 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.762431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44753 (local address 127.20.10.189:43165)
0807 22:43:46.806377 (+ 43946us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:46.806384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.806409 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:46.808314 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.808321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.808452 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.809245 (+ 793us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.814464 (+ 5219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.816871 (+ 2407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.823089 (+ 6218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.824243 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.824435 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.835188 (+ 10753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.835195 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.835199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.835249 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.841624 (+ 6375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.843088 (+ 1464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.843097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.843102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.843415 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.844275 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.844280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.844284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.844754 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.844941 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.846147 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.846501 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43774,"thread_start_us":43628,"threads_started":1}
I20260807 22:43:46.848963 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.763410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.20.10.130:44753)
0807 22:43:46.796737 (+ 33327us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:46.796842 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:46.807891 (+ 11049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:46.809399 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:46.809415 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.811127 (+ 1712us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.813815 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.813933 (+ 118us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.817287 (+ 3354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.817294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.822783 (+ 5489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.822805 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.825735 (+ 2930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.825746 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.826293 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.827509 (+ 1216us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.827567 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.834737 (+ 7170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.841896 (+ 7159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.841910 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.841935 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.842910 (+ 975us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.843626 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.843631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.843634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.844092 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.844913 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.844921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.845597 (+ 676us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.846004 (+ 407us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.846564 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32577,"mutex_wait_us":945,"spinlock_wait_cycles":2944,"thread_start_us":39484,"threads_started":1}
I20260807 22:43:46.850514 28492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.805137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.20.10.130:43973)
0807 22:43:46.808806 (+ 3669us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:46.808822 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:46.809012 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:46.809724 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:46.809728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.812429 (+ 2701us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.813995 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.814038 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.817249 (+ 3211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.817253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.825152 (+ 7899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.825177 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.827421 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.827436 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.827886 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.829904 (+ 2018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.829941 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.837351 (+ 7410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.844498 (+ 7147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.844504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.844507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.845505 (+ 998us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.847490 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.847496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.847499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.847835 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.849013 (+ 1178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.849017 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.849223 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.849355 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.849743 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3510,"mutex_wait_us":1014,"thread_start_us":3253,"threads_started":1}
I20260807 22:43:46.850925 28482 heartbeater.cc:347] Connected to a master server at 127.20.10.189:43165
I20260807 22:43:46.851325 28482 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.851642 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.762465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:58343 (local address 127.20.10.190:43009)
0807 22:43:46.807009 (+ 44544us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:46.807019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.807066 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:43:46.810257 (+ 3191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.810264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.810353 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.810487 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.815768 (+ 5281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.816924 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.822871 (+ 5947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.824040 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.824242 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.834037 (+ 9795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.834044 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.834047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.834089 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.842721 (+ 8632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.845036 (+ 2315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.845042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.845045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.845388 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.847206 (+ 1818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.847211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.847214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.847747 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.847888 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.848858 (+ 970us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.849755 (+ 897us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44350,"thread_start_us":44240,"threads_started":1}
I20260807 22:43:46.851854 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.803942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:43973 (local address 127.20.10.188:42771)
0807 22:43:46.804807 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:46.804813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.809132 (+ 4319us) server_negotiation.cc:407] Connection header received
0807 22:43:46.809206 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.809211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.809348 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.809465 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.814389 (+ 4924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.815535 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.825446 (+ 9911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.826644 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.826820 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.837723 (+ 10903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.837729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.837732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.837832 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.844292 (+ 6460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.845651 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.845657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.845660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.845886 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.848003 (+ 2117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.848009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.848013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.848683 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.848925 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.850913 (+ 1988us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.851172 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":606,"threads_started":1}
I20260807 22:43:46.852325 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.765371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:58343)
0807 22:43:46.801431 (+ 36060us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:46.801502 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:46.807906 (+ 6404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:46.810513 (+ 2607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:46.810518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:46.812429 (+ 1911us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:46.814262 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.814285 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.817236 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.817242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.822445 (+ 5203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:46.822483 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.825732 (+ 3249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.825759 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:46.826299 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.827702 (+ 1403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:46.827737 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:46.833649 (+ 5912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:46.843137 (+ 9488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.843142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.843145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.844190 (+ 1045us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.846789 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:46.846794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:46.846798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:46.847044 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:46.847962 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:46.847967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:46.848136 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 22:43:46.848378 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 22:43:46.851126 (+ 2748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35833,"mutex_wait_us":350,"spinlock_wait_cycles":27008,"thread_start_us":36335,"threads_started":1}
I20260807 22:43:46.853108 28482 heartbeater.cc:511] Master 127.20.10.189:43165 requested a full tablet report, sending...
I20260807 22:43:46.853669 28481 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:46.853983 28481 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.855218 28481 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
I20260807 22:43:46.855820 27914 ts_manager.cc:194] Registered new tserver with Master: cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223)
I20260807 22:43:46.856714 28480 heartbeater.cc:347] Connected to a master server at 127.20.10.188:42771
I20260807 22:43:46.856854 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:46.764684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.20.10.131:35225)
0807 22:43:46.800111 (+ 35427us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:46.800126 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:46.808363 (+ 8237us) server_negotiation.cc:407] Connection header received
0807 22:43:46.809020 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:46.809034 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:46.809358 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:46.809599 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:46.815894 (+ 6295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.819129 (+ 3235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.823412 (+ 4283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:46.824555 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:46.824781 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:46.838789 (+ 14008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:46.838827 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:46.838853 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:46.838906 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:46.845750 (+ 6844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.846937 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.846947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.846950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.847615 (+ 665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:46.848294 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:46.848299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:46.848302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:46.848923 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:46.849694 (+ 771us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:46.855760 (+ 6066us) server_negotiation.cc:299] Negotiation successful
0807 22:43:46.856056 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35069,"thread_start_us":39968,"threads_started":1}
I20260807 22:43:46.856947 28480 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:46.856989 27259 ts_manager.cc:194] Re-registered known tserver with Master: cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223)
I20260807 22:43:46.857780 28480 heartbeater.cc:511] Master 127.20.10.188:42771 requested a full tablet report, sending...
I20260807 22:43:46.858223 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:58343
I20260807 22:43:46.859622 28127 ts_manager.cc:194] Registered new tserver with Master: cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223)
I20260807 22:43:46.859725 27747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:46.954563 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27616
I20260807 22:43:46.978590 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:35225
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:43009
--webserver_port=33561
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:47.053579 28319 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
W20260807 22:43:47.368661 28494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:47.368993 28494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:47.369103 28494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:47.378641 28494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:47.378798 28494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:47.378925 28494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:47.390949 28494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:35225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=33561
--tserver_master_addrs=127.20.10.190:43009,127.20.10.189:43165,127.20.10.188:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:47.392581 28494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:47.394121 28494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:47.410971 28500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:47.411557 28499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:47.411091 28502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:47.412755 28494 server_base.cc:1034] running on GCE node
I20260807 22:43:47.413694 28494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:47.415408 28494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:47.416673 28494 hybrid_clock.cc:648] HybridClock initialized: now 1786142627416591 us; error 71 us; skew 500 ppm
I20260807 22:43:47.417110 28494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:47 dist-test-slave-csg5 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142627, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:47.421528 28494 init.cc:377] Logged in from keytab as oryx/127.20.10.131@KRBTEST.COM (short username oryx)
I20260807 22:43:47.423730 28494 webserver.cc:460] Webserver started at http://127.20.10.131:33561/ using document root <none> and password file <none>
I20260807 22:43:47.424559 28494 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:47.424754 28494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:47.431468 28494 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:47.435448 28509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:47.437940 28494 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:47.438180 28494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
format_stamp: "Formatted at 2026-08-07 22:43:43 on dist-test-slave-csg5"
server_key: "acd9d8b8ace261b9e30d4ddaaec5f91e"
server_key_iv: "d0eef07ad1bf7e5812a9745cd3f8a4db"
server_key_version: "encryptionkey@0"
I20260807 22:43:47.439572 28494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:47.475776 28494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:47.481168 28494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:47.481525 28494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:47.482803 28494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:47.484624 28494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:47.484754 28494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:47.484894 28494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:47.484988 28494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:47.524583 28494 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:35225
I20260807 22:43:47.524694 28629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:35225 every 8 connection(s)
I20260807 22:43:47.526139 28494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:47.530501 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 28494
Aug 07 22:43:47 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142627, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.189@KRBTEST.COM
Aug 07 22:43:47 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142627, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.188@KRBTEST.COM
Aug 07 22:43:47 dist-test-slave-csg5 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142627, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:47.576427 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.530770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.20.10.131:50093)
0807 22:43:47.531949 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.532007 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.534970 (+ 2963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.543467 (+ 8497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.543475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.544015 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:47.544760 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.544790 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.546467 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.546474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.549999 (+ 3525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.550017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.552225 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.552234 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.552737 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.554903 (+ 2166us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:47.554938 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:47.561726 (+ 6788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:47.569840 (+ 8114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.569848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.569854 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.571062 (+ 1208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.571804 (+ 742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.571808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.571811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.572185 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.574587 (+ 2402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:47.574591 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:47.574859 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.575004 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.575571 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":165,"thread_start_us":860,"threads_started":1}
I20260807 22:43:47.578161 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.533269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:43903 (local address 127.20.10.188:42771)
0807 22:43:47.539244 (+ 5975us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.539254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.543115 (+ 3861us) server_negotiation.cc:407] Connection header received
0807 22:43:47.543309 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.543314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.543399 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.543519 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.545114 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.546256 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.552975 (+ 6719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.554239 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.554382 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.562204 (+ 7822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.562212 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.562217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.562300 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.567816 (+ 5516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.569486 (+ 1670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.569494 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.569499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.569796 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.570638 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.570643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.570646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.571118 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.571288 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.577097 (+ 5809us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.577415 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5777,"thread_start_us":5633,"threads_started":1}
I20260807 22:43:47.575574 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.532836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.20.10.131:43903)
0807 22:43:47.542946 (+ 10110us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.542976 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.543154 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.543741 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.543746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.544405 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:47.544762 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.544790 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.546480 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.546488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.552737 (+ 6249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.552753 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.555046 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.555053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.555445 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.556739 (+ 1294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:47.556762 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:47.561763 (+ 5001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:47.568053 (+ 6290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.568065 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.568101 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.569227 (+ 1126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.569941 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.569951 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.569957 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.570461 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.571411 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:47.571423 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:47.572047 (+ 624us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.572529 (+ 482us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.573371 (+ 842us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9936,"mutex_wait_us":24,"thread_start_us":6633,"threads_started":1}
I20260807 22:43:47.579525 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.530318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:50093 (local address 127.20.10.189:43165)
0807 22:43:47.543108 (+ 12790us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.543117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.543139 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:43:47.543208 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.543217 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.543311 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.543441 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.545006 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.546263 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.550159 (+ 3896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.551512 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.551642 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.562093 (+ 10451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.562102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.562108 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.562210 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.569619 (+ 7409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.571228 (+ 1609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.571234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.571237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.571653 (+ 416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.572720 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.572726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.572729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.574359 (+ 1630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.574509 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.575827 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.576171 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12644,"thread_start_us":13180,"threads_started":1}
I20260807 22:43:47.581022 28630 heartbeater.cc:347] Connected to a master server at 127.20.10.188:42771
I20260807 22:43:47.581233 28632 heartbeater.cc:347] Connected to a master server at 127.20.10.189:43165
I20260807 22:43:47.581723 28632 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:47.581744 28630 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:47.584059 28630 heartbeater.cc:511] Master 127.20.10.188:42771 requested a full tablet report, sending...
I20260807 22:43:47.584085 28636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.529990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:43445 (local address 127.20.10.190:43009)
0807 22:43:47.530498 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.530505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.539960 (+ 9455us) server_negotiation.cc:407] Connection header received
0807 22:43:47.540351 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.540361 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.540477 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.540580 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.546934 (+ 6354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.548202 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.558145 (+ 9943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.559530 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.559732 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.569410 (+ 9678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.569419 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.569426 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.569474 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.577105 (+ 7631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.578542 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.578551 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.578558 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.578953 (+ 395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.579678 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.579684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.579697 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.580309 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.580454 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.582738 (+ 2284us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.583152 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":471,"threads_started":1}
I20260807 22:43:47.584197 28632 heartbeater.cc:511] Master 127.20.10.189:43165 requested a full tablet report, sending...
I20260807 22:43:47.584918 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.531527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:43445)
0807 22:43:47.539789 (+ 8262us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.539819 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.540143 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.541349 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.541370 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.542823 (+ 1453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:47.546730 (+ 3907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.546744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.548506 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.548514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.554421 (+ 5907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.554436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.560302 (+ 5866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.560312 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.562163 (+ 1851us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.563197 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:47.563229 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:47.568915 (+ 5686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:47.577383 (+ 8468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.577393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.577400 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.578324 (+ 924us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.579199 (+ 875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:47.579204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:47.579209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:47.579477 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:47.580531 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:47.580536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:47.581096 (+ 560us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.581418 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.581984 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8089,"mutex_wait_us":31,"thread_start_us":10947,"threads_started":1}
I20260807 22:43:47.587126 28127 ts_manager.cc:194] Registered new tserver with Master: 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:47.587324 27914 ts_manager.cc:194] Registered new tserver with Master: 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:47.587663 28631 heartbeater.cc:347] Connected to a master server at 127.20.10.190:43009
I20260807 22:43:47.587855 28631 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:47.588961 28631 heartbeater.cc:511] Master 127.20.10.190:43009 requested a full tablet report, sending...
I20260807 22:43:47.590507 27259 ts_manager.cc:194] Re-registered known tserver with Master: 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:47.592270 27259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:43445
I20260807 22:43:47.612098 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.547035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.20.10.190:43009)
0807 22:43:47.566958 (+ 19923us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.566968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.567033 (+ 65us) server_negotiation.cc:407] Connection header received
0807 22:43:47.567121 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.567130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.567261 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.567433 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.582813 (+ 15380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.585072 (+ 2259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.590142 (+ 5070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.591238 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.591399 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.598290 (+ 6891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.598302 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.598307 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.598362 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.605472 (+ 7110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.607046 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.607056 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.607059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.607516 (+ 457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.608325 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.608333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.608340 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.608983 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.609384 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.610117 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.610856 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17619,"thread_start_us":13124,"threads_started":1}
I20260807 22:43:47.620659 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.570323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.20.10.188:42771)
0807 22:43:47.578452 (+ 8129us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.578457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.589905 (+ 11448us) server_negotiation.cc:407] Connection header received
0807 22:43:47.590415 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.590423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.590739 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.590853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.598055 (+ 7202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.599918 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.603840 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.605290 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.605437 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.611788 (+ 6351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.611798 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.611804 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.611860 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.616702 (+ 4842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.617732 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.617739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.617743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.617940 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.618467 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.618472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.618476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.618991 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.619302 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.619816 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.620266 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8020,"thread_start_us":8831,"threads_started":1}
I20260807 22:43:47.621933 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.553999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.20.10.189:43165)
0807 22:43:47.561033 (+ 7034us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.561038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.590537 (+ 29499us) server_negotiation.cc:407] Connection header received
0807 22:43:47.590894 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.590900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.591026 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.591120 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:47.598409 (+ 7289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.600476 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.602796 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.604146 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.604308 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.610238 (+ 5930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:47.610248 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:47.610255 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:47.610309 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:47.616986 (+ 6677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.618433 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.618442 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.618445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.618753 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:47.619463 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:47.619469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:47.619472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:47.620084 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:47.620476 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.621326 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.621559 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6894,"thread_start_us":6779,"threads_started":1}
I20260807 22:43:47.627058 27259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:47.640076 27259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:47.688347 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.675522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41251 (local address 127.0.0.1:36686)
0807 22:43:47.678067 (+ 2545us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.678085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.678308 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.679123 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.679127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.679155 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:47.679809 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.679821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.681732 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.681738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.684753 (+ 3015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.684771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.687040 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.687049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.687319 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.687348 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.687469 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.687627 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2417,"thread_start_us":2194,"threads_started":1}
I20260807 22:43:47.688450 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.676005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.20.10.129:41251)
0807 22:43:47.677301 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.677324 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.678202 (+ 878us) server_negotiation.cc:407] Connection header received
0807 22:43:47.678565 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.678572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.678921 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.679143 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:47.680055 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.681530 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.684947 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.686558 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.686860 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.687186 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.687539 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.687769 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":832,"threads_started":1}
I20260807 22:43:47.695227 28249 tablet_service.cc:1467] Processing CreateTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee (DEFAULT_TABLE table=test-table [id=c021ffbfb95549e0bb13b7b6edb46408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:47.698455 28249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e3241755ff4c0387bba3db8f1db6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:47.706825 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.676184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38223 (local address 127.0.0.1:55466)
0807 22:43:47.688930 (+ 12746us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.688952 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.690259 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.694864 (+ 4605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.694869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.694899 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:47.695336 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.695349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.698537 (+ 3188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.698548 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.702160 (+ 3612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.702179 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.705142 (+ 2963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.705149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.705610 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.705659 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.705791 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.706020 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12624,"thread_start_us":13880,"threads_started":1}
I20260807 22:43:47.707460 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.676286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.20.10.130:38223)
0807 22:43:47.688692 (+ 12406us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.688708 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.689080 (+ 372us) server_negotiation.cc:407] Connection header received
0807 22:43:47.690569 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.690582 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.691110 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.691484 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:47.695651 (+ 4167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.698278 (+ 2627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.702421 (+ 4143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.704873 (+ 2452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.705094 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.705411 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.706707 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.706975 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12069,"thread_start_us":12469,"threads_started":1}
I20260807 22:43:47.713739 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.690944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35225 (local address 127.0.0.1:54476)
0807 22:43:47.692713 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:47.692766 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:47.692904 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:47.699012 (+ 6108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:47.699017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:47.699054 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:47.700460 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.700500 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.704136 (+ 3636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.704141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.708120 (+ 3979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:47.708146 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.711513 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.711523 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:47.711906 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.711945 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:43:47.712060 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:43:47.712252 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1643,"mutex_wait_us":145}
I20260807 22:43:47.714979 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:47.691612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.20.10.131:35225)
0807 22:43:47.697316 (+ 5704us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:47.697326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:47.697392 (+ 66us) server_negotiation.cc:407] Connection header received
0807 22:43:47.697508 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:47.697517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:47.698759 (+ 1242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:47.699418 (+ 659us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:47.700948 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.703893 (+ 2945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.708329 (+ 4436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:47.710589 (+ 2260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:47.710793 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:47.711144 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:47.714085 (+ 2941us) server_negotiation.cc:299] Negotiation successful
0807 22:43:47.714367 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5385,"thread_start_us":5016,"threads_started":1}
I20260807 22:43:47.714149 28405 tablet_service.cc:1467] Processing CreateTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee (DEFAULT_TABLE table=test-table [id=c021ffbfb95549e0bb13b7b6edb46408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:47.716185 28405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e3241755ff4c0387bba3db8f1db6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:47.721918 28560 tablet_service.cc:1467] Processing CreateTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee (DEFAULT_TABLE table=test-table [id=c021ffbfb95549e0bb13b7b6edb46408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:47.724746 28560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e3241755ff4c0387bba3db8f1db6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:47.725162 28670 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Bootstrap starting.
I20260807 22:43:47.734285 28670 tablet_bootstrap.cc:654] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:47.736961 28670 log.cc:824] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:47.742252 28670 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: No bootstrap required, opened a new log
I20260807 22:43:47.742843 28670 ts_tablet_manager.cc:1397] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.005s
I20260807 22:43:47.749894 28672 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Bootstrap starting.
I20260807 22:43:47.755581 28674 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Bootstrap starting.
I20260807 22:43:47.760718 28672 tablet_bootstrap.cc:654] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:47.762151 28670 raft_consensus.cc:348] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.763231 28670 raft_consensus.cc:374] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:47.763528 28670 raft_consensus.cc:729] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ee3502329748c1af23a06553254d6e, State: Initialized, Role: FOLLOWER
I20260807 22:43:47.763852 28672 log.cc:824] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:47.764568 28674 tablet_bootstrap.cc:654] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:47.764935 28670 consensus_queue.cc:260] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.767949 28674 log.cc:824] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:47.784946 28672 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: No bootstrap required, opened a new log
I20260807 22:43:47.786072 28672 ts_tablet_manager.cc:1397] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.022s
I20260807 22:43:47.787415 28670 ts_tablet_manager.cc:1428] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Time spent starting tablet: real 0.044s user 0.014s sys 0.025s
I20260807 22:43:47.789255 28674 tablet_bootstrap.cc:492] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: No bootstrap required, opened a new log
I20260807 22:43:47.789746 28674 ts_tablet_manager.cc:1397] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.020s
I20260807 22:43:47.804921 28672 raft_consensus.cc:348] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.805512 28672 raft_consensus.cc:374] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:47.805127 28674 raft_consensus.cc:348] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.805663 28672 raft_consensus.cc:729] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa9a9ee55574cbc8958c35681cca9ad, State: Initialized, Role: FOLLOWER
I20260807 22:43:47.805886 28674 raft_consensus.cc:374] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:47.806125 28674 raft_consensus.cc:729] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe4e3dfdec4bc6ad353e805e0cf8f9, State: Initialized, Role: FOLLOWER
I20260807 22:43:47.806481 28672 consensus_queue.cc:260] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.807152 28674 consensus_queue.cc:260] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:47.809803 28672 ts_tablet_manager.cc:1428] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Time spent starting tablet: real 0.023s user 0.008s sys 0.013s
I20260807 22:43:47.810029 28674 ts_tablet_manager.cc:1428] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Time spent starting tablet: real 0.020s user 0.019s sys 0.003s
I20260807 22:43:47.810279 28481 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
I20260807 22:43:47.810551 28631 heartbeater.cc:503] Master 127.20.10.190:43009 was elected leader, sending a full tablet report...
W20260807 22:43:47.991156 28321 tablet.cc:2369] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:48.020902 28483 tablet.cc:2369] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:48.030507 28634 tablet.cc:2369] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:48.041759 28676 raft_consensus.cc:482] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:48.042348 28676 raft_consensus.cc:504] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:48.045212 28676 leader_election.cc:288] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223), 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:48.056562 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.045523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:55677 (local address 127.20.10.130:38223)
0807 22:43:48.045794 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.045800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.047289 (+ 1489us) server_negotiation.cc:407] Connection header received
0807 22:43:48.047639 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.047649 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.047775 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.047859 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:48.049330 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.050460 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.053953 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.055252 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.055459 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.055571 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.056021 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.056158 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:43:48.056874 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.047386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48685 (local address 127.20.10.131:35225)
0807 22:43:48.047653 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.047662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.048443 (+ 781us) server_negotiation.cc:407] Connection header received
0807 22:43:48.048711 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.048716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.048790 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.048902 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:48.049884 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.051131 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.054086 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.055479 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.055695 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.055808 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.056343 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.056505 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 22:43:48.057232 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.045521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38223 (local address 127.20.10.129:55677)
0807 22:43:48.047151 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:48.047172 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:48.047381 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:48.048126 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:48.048129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:48.048153 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:48.049102 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:48.049118 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.050822 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.050826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:48.053792 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:48.053806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.055631 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.055644 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:48.055840 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.055866 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:43:48.055991 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 22:43:48.056143 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1363,"mutex_wait_us":20,"spinlock_wait_cycles":56576,"thread_start_us":969,"threads_started":1}
I20260807 22:43:48.057634 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.047139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35225 (local address 127.20.10.129:48685)
0807 22:43:48.048317 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:48.048335 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:48.048536 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:48.048923 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:48.048928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:48.048957 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:48.049653 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:48.049665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.051278 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.051300 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:48.053880 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:48.053895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.055980 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.055987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:48.056144 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.056175 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:43:48.056299 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 22:43:48.056447 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"mutex_wait_us":1,"thread_start_us":712,"threads_started":1}
I20260807 22:43:48.058689 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e3241755ff4c0387bba3db8f1db6ee" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" is_pre_election: true
I20260807 22:43:48.059628 28430 raft_consensus.cc:2454] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 0.
I20260807 22:43:48.059520 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e3241755ff4c0387bba3db8f1db6ee" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" is_pre_election: true
I20260807 22:43:48.060673 28582 raft_consensus.cc:2454] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 0.
I20260807 22:43:48.061100 28204 leader_election.cc:302] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81ee3502329748c1af23a06553254d6e, cfa9a9ee55574cbc8958c35681cca9ad; no voters:
I20260807 22:43:48.061810 28676 raft_consensus.cc:2781] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:48.062222 28676 raft_consensus.cc:482] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:48.062410 28676 raft_consensus.cc:3037] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:48.066424 28676 raft_consensus.cc:504] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:48.067474 28676 leader_election.cc:288] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 election: Requested vote from peers cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223), 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:48.068284 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e3241755ff4c0387bba3db8f1db6ee" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
I20260807 22:43:48.068382 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e3241755ff4c0387bba3db8f1db6ee" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
I20260807 22:43:48.068606 28582 raft_consensus.cc:3037] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:48.068730 28430 raft_consensus.cc:3037] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:48.072867 28582 raft_consensus.cc:2454] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 1.
I20260807 22:43:48.072867 28430 raft_consensus.cc:2454] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 1.
I20260807 22:43:48.073711 28204 leader_election.cc:302] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81ee3502329748c1af23a06553254d6e, cfa9a9ee55574cbc8958c35681cca9ad; no voters:
I20260807 22:43:48.074203 28676 raft_consensus.cc:2781] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:48.075501 28676 raft_consensus.cc:686] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 1 LEADER]: Becoming Leader. State: Replica: 81ee3502329748c1af23a06553254d6e, State: Running, Role: LEADER
I20260807 22:43:48.076447 28676 consensus_queue.cc:237] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:48.083847 27259 catalog_manager.cc:5495] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 81ee3502329748c1af23a06553254d6e (127.20.10.129). New cstate: current_term: 1 leader_uuid: "81ee3502329748c1af23a06553254d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:48.169567 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.160752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.20.10.129:41251)
0807 22:43:48.161032 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.161037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.161346 (+ 309us) server_negotiation.cc:407] Connection header received
0807 22:43:48.161468 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.161478 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.161606 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.161796 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:48.162276 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.163522 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.166027 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.166975 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.167105 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.168021 (+ 916us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:48.168669 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:48.168886 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.169063 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.169151 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:43:48.180493 28430 raft_consensus.cc:1261] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Refusing update from remote peer 81ee3502329748c1af23a06553254d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:48.180586 28582 raft_consensus.cc:1261] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Refusing update from remote peer 81ee3502329748c1af23a06553254d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:48.182217 28683 consensus_queue.cc:1035] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 22:43:48.183094 28676 consensus_queue.cc:1035] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:48.205000 28688 mvcc.cc:205] Tried to move back new op lower bound from 7316040205013811200 to 7316040204606943232. Current Snapshot: MvccSnapshot[applied={T|T < 7316040205013811200}]
I20260807 22:43:48.211898 28689 mvcc.cc:205] Tried to move back new op lower bound from 7316040205013811200 to 7316040204606943232. Current Snapshot: MvccSnapshot[applied={T|T < 7316040205013811200}]
I20260807 22:43:48.233536 27258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:48.233949 27258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:48.244776 27258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f36bc52d5b1432283883efed09e4aea: Sending DeleteTablet for 3 replicas of tablet d9e3241755ff4c0387bba3db8f1db6ee
I20260807 22:43:48.247231 28249 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:48 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:36686
I20260807 22:43:48.247476 28405 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:48 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:55466
I20260807 22:43:48.247438 28560 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e3241755ff4c0387bba3db8f1db6ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:48 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:54476
I20260807 22:43:48.282097 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27223
I20260807 22:43:48.285991 27293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:48.292356 28696 tablet_replica.cc:330] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: stopping tablet replica
I20260807 22:43:48.293138 28696 raft_consensus.cc:2229] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:48.293932 28698 tablet_replica.cc:330] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: stopping tablet replica
I20260807 22:43:48.294602 28698 raft_consensus.cc:2229] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:48.295948 28698 raft_consensus.cc:2258] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:48.298882 28697 tablet_replica.cc:330] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: stopping tablet replica
I20260807 22:43:48.299479 28696 raft_consensus.cc:2258] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:48.299566 28697 raft_consensus.cc:2229] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:48.300575 28697 raft_consensus.cc:2258] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:48.302517 28696 ts_tablet_manager.cc:1905] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:48.306903 28698 ts_tablet_manager.cc:1905] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:48.306911 28697 ts_tablet_manager.cc:1905] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:48.323398 28698 ts_tablet_manager.cc:1918] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:48.323800 28698 log.cc:1196] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/d9e3241755ff4c0387bba3db8f1db6ee
I20260807 22:43:48.325158 28698 ts_tablet_manager.cc:1939] T d9e3241755ff4c0387bba3db8f1db6ee P cfa9a9ee55574cbc8958c35681cca9ad: Deleting consensus metadata
I20260807 22:43:48.329093 27244 catalog_manager.cc:4841] TS cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223): tablet d9e3241755ff4c0387bba3db8f1db6ee (table test-table [id=c021ffbfb95549e0bb13b7b6edb46408]) successfully deleted
I20260807 22:43:48.336298 28696 ts_tablet_manager.cc:1918] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:48.336828 28696 log.cc:1196] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/d9e3241755ff4c0387bba3db8f1db6ee
I20260807 22:43:48.338997 28696 ts_tablet_manager.cc:1939] T d9e3241755ff4c0387bba3db8f1db6ee P 81ee3502329748c1af23a06553254d6e: Deleting consensus metadata
I20260807 22:43:48.347312 28697 ts_tablet_manager.cc:1918] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:48.352895 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.251265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.20.10.189:43165)
0807 22:43:48.292048 (+ 40783us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.292057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.292089 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:43:48.292186 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.292196 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.292392 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.293536 (+ 1144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:48.297884 (+ 4348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.299770 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.303934 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.304991 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.305149 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.326089 (+ 20940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:48.326102 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:48.326110 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:48.326220 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:48.334806 (+ 8586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.347160 (+ 12354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.347168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.347175 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.347590 (+ 415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.348679 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.348688 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.348694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.349407 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:48.349863 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.351548 (+ 1685us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.351972 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40564,"thread_start_us":40354,"threads_started":1}
I20260807 22:43:48.348012 28697 log.cc:1196] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/d9e3241755ff4c0387bba3db8f1db6ee
I20260807 22:43:48.357043 28697 ts_tablet_manager.cc:1939] T d9e3241755ff4c0387bba3db8f1db6ee P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting consensus metadata
I20260807 22:43:48.401635 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.281277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.20.10.188:42771)
0807 22:43:48.285225 (+ 3948us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.285235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.356136 (+ 70901us) server_negotiation.cc:407] Connection header received
0807 22:43:48.356537 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.356547 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.356707 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.356856 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:48.360459 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.362840 (+ 2381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.377799 (+ 14959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.379348 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.379518 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.386980 (+ 7462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:48.386990 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:48.386996 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:48.387056 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:48.395318 (+ 8262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.396817 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.396825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.396830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.397279 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.398024 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.398030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.398034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.398570 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:48.399357 (+ 787us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.400462 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.400848 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3746,"thread_start_us":2642,"threads_started":1}
I20260807 22:43:48.593295 27242 catalog_manager.cc:4841] TS 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251): tablet d9e3241755ff4c0387bba3db8f1db6ee (table test-table [id=c021ffbfb95549e0bb13b7b6edb46408]) successfully deleted
I20260807 22:43:48.593827 27244 catalog_manager.cc:4841] TS 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225): tablet d9e3241755ff4c0387bba3db8f1db6ee (table test-table [id=c021ffbfb95549e0bb13b7b6edb46408]) successfully deleted
I20260807 22:43:48.595610 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27223
I20260807 22:43:48.674991 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.648511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.20.10.189:43165)
0807 22:43:48.648707 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.648715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.651102 (+ 2387us) server_negotiation.cc:407] Connection header received
0807 22:43:48.651448 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.651454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.651546 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.651628 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:48.654184 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.655431 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.657524 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.658213 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.658330 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.662577 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:48.662587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:48.662596 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:48.662689 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:48.668669 (+ 5980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.670447 (+ 1778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.670456 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.670462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.670791 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.671527 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.671534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.671539 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.671987 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:48.672485 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.673987 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.674319 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:43:48.681843 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:48.660427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.20.10.188:42771)
0807 22:43:48.660657 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:48.660665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:48.662795 (+ 2130us) server_negotiation.cc:407] Connection header received
0807 22:43:48.663071 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:48.663078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:48.663169 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:48.663244 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:48.665280 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.666468 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.668474 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:48.669167 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:48.669273 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:48.673074 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:48.673081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:48.673086 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:48.673133 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:48.677169 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.678463 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.678469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.678473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.678690 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:48.679261 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:48.679267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:48.679273 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:48.679912 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:48.680329 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:48.681169 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 22:43:48.681466 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:43:49.214883 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.214038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:58099)
0807 22:43:49.214513 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.214639 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":257,"threads_started":1}
W20260807 22:43:49.215565 28631 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:43009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.219444 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.218481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:60449)
0807 22:43:49.219023 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.219233 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":320,"threads_started":1}
W20260807 22:43:49.219952 28319 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:43009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.224676 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.223720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:56475)
0807 22:43:49.224345 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.224506 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":277,"threads_started":1}
W20260807 22:43:49.225319 28481 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:43009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.743906 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f36bc52d5b1432283883efed09e4aea)
I20260807 22:43:49.744469 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:49.745846 28715 raft_consensus.cc:482] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f36bc52d5b1432283883efed09e4aea)
I20260807 22:43:49.746188 28715 raft_consensus.cc:504] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:49.746456 28714 leader_election.cc:288] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189:43165)
I20260807 22:43:49.757907 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.748605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58134)
0807 22:43:49.757260 (+ 8655us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.757546 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8431,"threads_started":1}
W20260807 22:43:49.761396 27898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.762295 28715 leader_election.cc:288] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188:42771)
I20260807 22:43:49.768611 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.746730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58128)
0807 22:43:49.767714 (+ 20984us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.768010 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13238,"thread_start_us":17438,"threads_started":1}
I20260807 22:43:49.769201 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.768614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58162)
0807 22:43:49.768904 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.769013 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260807 22:43:49.769089 28113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
W20260807 22:43:49.769574 27898 leader_election.cc:334] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.771700 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.771120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58174)
0807 22:43:49.771390 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.771497 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260807 22:43:49.772126 28113 leader_election.cc:334] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.778293 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.746899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:43165 (local address 127.0.0.1:53622)
0807 22:43:49.766885 (+ 19986us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.766913 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:49.767052 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:49.769156 (+ 2104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:49.769164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:49.769204 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:49.769829 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:49.769851 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.772404 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.772410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:49.774941 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:49.774960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.777085 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.777094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:49.777424 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:49.777465 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:43:49.777585 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 22:43:49.777765 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19775,"mutex_wait_us":49,"thread_start_us":20474,"threads_started":1}
I20260807 22:43:49.778239 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.754378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:42771 (local address 127.0.0.1:33022)
0807 22:43:49.757546 (+ 3168us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.757566 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:49.757743 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:49.766811 (+ 9068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:49.766822 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:49.766893 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:49.770791 (+ 3898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:49.770811 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.772950 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.772956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:49.775263 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:49.775289 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.777200 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.777208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:49.777436 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:49.777471 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:43:49.777576 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:43:49.777746 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3022,"thread_start_us":2388,"threads_started":1}
I20260807 22:43:49.778430 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.746902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.20.10.189:43165)
0807 22:43:49.768713 (+ 21811us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:49.768721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:49.768749 (+ 28us) server_negotiation.cc:407] Connection header received
0807 22:43:49.768833 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:49.768838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:49.768923 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:49.770067 (+ 1144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:49.770362 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.772210 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.775187 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.776792 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.776983 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:49.777137 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:49.777642 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 22:43:49.777822 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21557,"thread_start_us":20679,"threads_started":1}
I20260807 22:43:49.778383 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.755227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.20.10.188:42771)
0807 22:43:49.764432 (+ 9205us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:49.764439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:49.764466 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:49.764552 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:49.764559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:49.764639 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:49.764787 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:49.771191 (+ 6404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.772783 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.775467 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:49.776945 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:49.777105 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:49.777283 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:49.777647 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 22:43:49.777926 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":9023,"thread_start_us":8849,"threads_started":1}
I20260807 22:43:49.779767 28140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d231b5f21b2e409599df65ad0ca5d224" is_pre_election: true
I20260807 22:43:49.780009 27924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d231b5f21b2e409599df65ad0ca5d224" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" is_pre_election: true
I20260807 22:43:49.780340 28140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de12ce6fb83b4c07ad867beb0fb35bb0 in term 1.
I20260807 22:43:49.780467 27924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d231b5f21b2e409599df65ad0ca5d224 in term 1.
I20260807 22:43:49.781101 27899 leader_election.cc:302] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d231b5f21b2e409599df65ad0ca5d224, de12ce6fb83b4c07ad867beb0fb35bb0; no voters: 6f36bc52d5b1432283883efed09e4aea
I20260807 22:43:49.781102 28114 leader_election.cc:302] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d231b5f21b2e409599df65ad0ca5d224, de12ce6fb83b4c07ad867beb0fb35bb0; no voters: 6f36bc52d5b1432283883efed09e4aea
I20260807 22:43:49.781651 28715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:43:49.781662 28714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:43:49.781814 28715 raft_consensus.cc:482] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6f36bc52d5b1432283883efed09e4aea)
I20260807 22:43:49.781819 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6f36bc52d5b1432283883efed09e4aea)
I20260807 22:43:49.781944 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:43:49.781944 28715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:43:49.785777 28715 raft_consensus.cc:504] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:49.785776 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:49.786736 28715 leader_election.cc:288] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 election: Requested vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), d231b5f21b2e409599df65ad0ca5d224 (127.20.10.188:42771)
I20260807 22:43:49.786778 28714 leader_election.cc:288] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 election: Requested vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189:43165)
I20260807 22:43:49.787446 28140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d231b5f21b2e409599df65ad0ca5d224"
I20260807 22:43:49.787643 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.787033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58186)
0807 22:43:49.787273 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.787394 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 22:43:49.787897 28140 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate de12ce6fb83b4c07ad867beb0fb35bb0 in current term 2: Already voted for candidate d231b5f21b2e409599df65ad0ca5d224 in this term.
I20260807 22:43:49.787784 27924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d231b5f21b2e409599df65ad0ca5d224" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
I20260807 22:43:49.788107 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.787350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58200)
0807 22:43:49.787616 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.787759 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260807 22:43:49.788194 27924 raft_consensus.cc:2379] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d231b5f21b2e409599df65ad0ca5d224 in current term 2: Already voted for candidate de12ce6fb83b4c07ad867beb0fb35bb0 in this term.
I20260807 22:43:49.788944 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.788420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58210)
0807 22:43:49.788607 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.788742 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 22:43:49.789245 28113 leader_election.cc:334] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.789386 28113 leader_election.cc:302] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d231b5f21b2e409599df65ad0ca5d224; no voters: 6f36bc52d5b1432283883efed09e4aea, de12ce6fb83b4c07ad867beb0fb35bb0
I20260807 22:43:49.789583 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:49.789215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58222)
0807 22:43:49.789378 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:49.789457 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 22:43:49.789711 28714 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 22:43:49.789767 27898 leader_election.cc:334] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:49.789924 27898 leader_election.cc:302] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de12ce6fb83b4c07ad867beb0fb35bb0; no voters: 6f36bc52d5b1432283883efed09e4aea, d231b5f21b2e409599df65ad0ca5d224
I20260807 22:43:49.790259 28715 raft_consensus.cc:2726] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 22:43:50.218084 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:50.217036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:47313)
0807 22:43:50.217649 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:50.217856 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":333,"threads_started":1}
I20260807 22:43:50.221730 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:50.220817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:40733)
0807 22:43:50.221415 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:50.221518 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":327,"threads_started":1}
I20260807 22:43:50.227440 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:50.226329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:45101)
0807 22:43:50.227023 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:50.227151 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":432,"threads_started":1}
I20260807 22:43:51.220233 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.219360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:55241)
0807 22:43:51.219892 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.220005 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":280,"threads_started":1}
I20260807 22:43:51.223579 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.222730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:39621)
0807 22:43:51.223324 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.223414 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":309,"threads_started":1}
I20260807 22:43:51.231034 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.228770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:35849)
0807 22:43:51.230732 (+ 1962us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.230837 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1768,"thread_start_us":1621,"threads_started":1}
I20260807 22:43:51.353022 28733 raft_consensus.cc:482] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:51.353397 28733 raft_consensus.cc:504] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:51.354139 28733 leader_election.cc:288] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189:43165)
I20260807 22:43:51.354835 27924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d231b5f21b2e409599df65ad0ca5d224" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" is_pre_election: true
I20260807 22:43:51.355175 27924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d231b5f21b2e409599df65ad0ca5d224 in term 2.
I20260807 22:43:51.355549 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.354439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58490)
0807 22:43:51.355154 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.355294 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":424,"threads_started":1}
I20260807 22:43:51.355794 28114 leader_election.cc:302] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d231b5f21b2e409599df65ad0ca5d224, de12ce6fb83b4c07ad867beb0fb35bb0; no voters:
I20260807 22:43:51.356217 28733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 22:43:51.356390 28733 raft_consensus.cc:482] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:51.356547 28733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:43:51.357609 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.357098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58500)
0807 22:43:51.357313 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.357431 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 22:43:51.357972 28113 leader_election.cc:334] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:51.359594 28733 raft_consensus.cc:504] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:51.360294 28733 leader_election.cc:288] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 election: Requested vote from peers 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009), de12ce6fb83b4c07ad867beb0fb35bb0 (127.20.10.189:43165)
I20260807 22:43:51.360916 27924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d231b5f21b2e409599df65ad0ca5d224" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0"
I20260807 22:43:51.361089 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.360498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58502)
0807 22:43:51.360699 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.360853 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 22:43:51.361205 27924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:43:51.362283 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.361864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58506)
0807 22:43:51.362040 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.362156 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 22:43:51.362524 28113 leader_election.cc:334] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
I20260807 22:43:51.363718 27924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d231b5f21b2e409599df65ad0ca5d224 in term 3.
I20260807 22:43:51.364249 28114 leader_election.cc:302] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d231b5f21b2e409599df65ad0ca5d224, de12ce6fb83b4c07ad867beb0fb35bb0; no voters: 6f36bc52d5b1432283883efed09e4aea
I20260807 22:43:51.364599 28733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:43:51.365145 28733 raft_consensus.cc:686] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [term 3 LEADER]: Becoming Leader. State: Replica: d231b5f21b2e409599df65ad0ca5d224, State: Running, Role: LEADER
I20260807 22:43:51.365650 28733 consensus_queue.cc:237] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } }
I20260807 22:43:51.367995 28736 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d231b5f21b2e409599df65ad0ca5d224. Latest consensus state: current_term: 3 leader_uuid: "d231b5f21b2e409599df65ad0ca5d224" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:51.368556 28736 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:51.369524 28738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:51.374238 28738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:51.375360 28738 catalog_manager.cc:1259] Loaded cluster ID: ace5f31a27fa4c9cb79d54f2f3af9199
I20260807 22:43:51.375456 28738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:51.376856 28738 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:51.377832 28738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Loaded TSK: 0
I20260807 22:43:51.378890 28738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:51.398618 28126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:51.407809 28126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:51.413926 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.413351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58524)
0807 22:43:51.413612 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.413735 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 22:43:51.414047 27924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [term 3 FOLLOWER]: Refusing update from remote peer d231b5f21b2e409599df65ad0ca5d224: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 22:43:51.414969 28736 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 22:43:51.415395 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.414878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58534)
0807 22:43:51.415114 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.415236 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:43:51.415858 28113 consensus_peers.cc:487] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 -> Peer 6f36bc52d5b1432283883efed09e4aea (127.20.10.190:43009): Couldn't send request to peer 6f36bc52d5b1432283883efed09e4aea. Status: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:51.418879 28740 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d231b5f21b2e409599df65ad0ca5d224. Latest consensus state: current_term: 3 leader_uuid: "d231b5f21b2e409599df65ad0ca5d224" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:51.419255 28740 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:51.421074 28733 sys_catalog.cc:455] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d231b5f21b2e409599df65ad0ca5d224" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:51.421550 28733 sys_catalog.cc:458] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:51.426246 28740 sys_catalog.cc:455] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d231b5f21b2e409599df65ad0ca5d224" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6f36bc52d5b1432283883efed09e4aea" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 43009 } } peers { permanent_uuid: "de12ce6fb83b4c07ad867beb0fb35bb0" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 43165 } attrs { promote: false } } peers { permanent_uuid: "d231b5f21b2e409599df65ad0ca5d224" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 42771 } attrs { promote: false } } }
I20260807 22:43:51.426570 28740 sys_catalog.cc:458] T 00000000000000000000000000000000 P de12ce6fb83b4c07ad867beb0fb35bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:51.448094 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.434212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38223 (local address 127.0.0.1:55480)
0807 22:43:51.435066 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.435091 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:51.435235 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:51.436321 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:51.436330 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:51.436371 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:51.436907 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.436923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.439277 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.439287 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.443159 (+ 3872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.443194 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.446587 (+ 3393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.446597 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.446991 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.447055 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 22:43:51.447187 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 22:43:51.447409 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":30,"thread_start_us":1004,"threads_started":1}
I20260807 22:43:51.449052 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.434772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.20.10.130:38223)
0807 22:43:51.435568 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.435574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.435593 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:43:51.435803 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.435810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.435896 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.436022 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:51.437199 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.439000 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.443845 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.445992 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.446164 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.446331 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.448455 (+ 2124us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.448647 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":340,"threads_started":1}
I20260807 22:43:51.450237 28405 tablet_service.cc:1467] Processing CreateTablet for tablet f50fbc054be4427397531b7a82e500a0 (DEFAULT_TABLE table=test-table [id=0f2d4787ec184678957131d7c28a5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:51.450861 28405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50fbc054be4427397531b7a82e500a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:51.451071 28748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.435651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35225 (local address 127.0.0.1:54486)
0807 22:43:51.436490 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.436520 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:51.436695 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:51.443809 (+ 7114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:51.443815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:51.443850 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:51.444182 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.444190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.446117 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.446123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.448084 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.448100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.450140 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.450148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.450343 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.450378 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:43:51.450451 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:43:51.450608 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":46,"thread_start_us":340,"threads_started":1}
I20260807 22:43:51.452210 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.435901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54486 (local address 127.20.10.131:35225)
0807 22:43:51.443377 (+ 7476us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.443383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.443408 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:51.443494 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.443499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.443595 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.444941 (+ 1346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:51.445177 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.445958 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.448473 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.449882 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.451475 (+ 1593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.451604 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.451664 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.451830 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7300,"thread_start_us":7116,"threads_started":1}
I20260807 22:43:51.453179 28560 tablet_service.cc:1467] Processing CreateTablet for tablet f50fbc054be4427397531b7a82e500a0 (DEFAULT_TABLE table=test-table [id=0f2d4787ec184678957131d7c28a5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:51.453666 28560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50fbc054be4427397531b7a82e500a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:51.455585 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.433535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41251 (local address 127.0.0.1:36702)
0807 22:43:51.433719 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.433750 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:51.434049 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:51.448984 (+ 14935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:51.448989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:51.449026 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:51.449374 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.449385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.451493 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.451499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.453425 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:51.453442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.454796 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.454805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:51.454979 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.455003 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:43:51.455112 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 22:43:51.455259 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:43:51.455670 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.433689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.20.10.129:41251)
0807 22:43:51.448252 (+ 14563us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.448260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.448591 (+ 331us) server_negotiation.cc:407] Connection header received
0807 22:43:51.448681 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.448694 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.448803 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.449873 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:51.450126 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.451343 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.453599 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.454504 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.454701 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.454849 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.455154 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.455312 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14339,"thread_start_us":14742,"threads_started":1}
I20260807 22:43:51.456425 28249 tablet_service.cc:1467] Processing CreateTablet for tablet f50fbc054be4427397531b7a82e500a0 (DEFAULT_TABLE table=test-table [id=0f2d4787ec184678957131d7c28a5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:51.456897 28249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50fbc054be4427397531b7a82e500a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:51.457819 28750 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Bootstrap starting.
I20260807 22:43:51.459949 28750 tablet_bootstrap.cc:654] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:51.460495 28751 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Bootstrap starting.
I20260807 22:43:51.462389 28751 tablet_bootstrap.cc:654] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:51.462747 28750 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: No bootstrap required, opened a new log
I20260807 22:43:51.462947 28750 ts_tablet_manager.cc:1397] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:51.463809 28750 raft_consensus.cc:348] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.464108 28750 raft_consensus.cc:374] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:51.464197 28750 raft_consensus.cc:729] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa9a9ee55574cbc8958c35681cca9ad, State: Initialized, Role: FOLLOWER
I20260807 22:43:51.464608 28751 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: No bootstrap required, opened a new log
I20260807 22:43:51.464450 28750 consensus_queue.cc:260] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.464775 28751 ts_tablet_manager.cc:1397] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 22:43:51.465615 28750 ts_tablet_manager.cc:1428] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:43:51.465685 28751 raft_consensus.cc:348] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.465951 28751 raft_consensus.cc:374] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:51.466063 28751 raft_consensus.cc:729] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fe4e3dfdec4bc6ad353e805e0cf8f9, State: Initialized, Role: FOLLOWER
I20260807 22:43:51.466349 28751 consensus_queue.cc:260] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.467162 28755 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Bootstrap starting.
I20260807 22:43:51.467222 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.466657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:55015)
0807 22:43:51.466945 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.467053 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 22:43:51.469995 28755 tablet_bootstrap.cc:654] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:51.470197 28480 heartbeater.cc:503] Master 127.20.10.188:42771 was elected leader, sending a full tablet report...
I20260807 22:43:51.471650 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.470963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:33171)
0807 22:43:51.471251 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.471377 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 22:43:51.473682 28751 ts_tablet_manager.cc:1428] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Time spent starting tablet: real 0.009s user 0.000s sys 0.009s
I20260807 22:43:51.474299 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.473864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:52681)
0807 22:43:51.474042 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.474146 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:43:51.475661 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.475038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:42285)
0807 22:43:51.475245 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.475355 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 22:43:51.478560 28630 heartbeater.cc:503] Master 127.20.10.188:42771 was elected leader, sending a full tablet report...
I20260807 22:43:51.480168 28755 tablet_bootstrap.cc:492] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: No bootstrap required, opened a new log
I20260807 22:43:51.480367 28755 ts_tablet_manager.cc:1397] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260807 22:43:51.481212 28755 raft_consensus.cc:348] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.481428 28755 raft_consensus.cc:374] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:51.481493 28755 raft_consensus.cc:729] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ee3502329748c1af23a06553254d6e, State: Initialized, Role: FOLLOWER
I20260807 22:43:51.481729 28755 consensus_queue.cc:260] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.482556 28755 ts_tablet_manager.cc:1428] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260807 22:43:51.483515 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.482982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:58681)
0807 22:43:51.483144 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.483249 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:43:51.484732 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.484223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:46513)
0807 22:43:51.484473 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.484563 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 22:43:51.484755 28318 heartbeater.cc:503] Master 127.20.10.188:42771 was elected leader, sending a full tablet report...
I20260807 22:43:51.518998 28759 raft_consensus.cc:482] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:51.519261 28759 raft_consensus.cc:504] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.520279 28759 leader_election.cc:288] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223), 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:51.521162 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50fbc054be4427397531b7a82e500a0" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" is_pre_election: true
I20260807 22:43:51.521184 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50fbc054be4427397531b7a82e500a0" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" is_pre_election: true
I20260807 22:43:51.521461 28430 raft_consensus.cc:2454] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 0.
I20260807 22:43:51.521471 28582 raft_consensus.cc:2454] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 0.
I20260807 22:43:51.522073 28204 leader_election.cc:302] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81ee3502329748c1af23a06553254d6e, 93fe4e3dfdec4bc6ad353e805e0cf8f9; no voters:
I20260807 22:43:51.522459 28759 raft_consensus.cc:2781] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:51.522710 28759 raft_consensus.cc:482] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:51.522836 28759 raft_consensus.cc:3037] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:51.525954 28759 raft_consensus.cc:504] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.527174 28759 leader_election.cc:288] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 election: Requested vote from peers cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223), 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225)
I20260807 22:43:51.527390 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50fbc054be4427397531b7a82e500a0" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfa9a9ee55574cbc8958c35681cca9ad"
I20260807 22:43:51.527668 28430 raft_consensus.cc:3037] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:51.527654 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f50fbc054be4427397531b7a82e500a0" candidate_uuid: "81ee3502329748c1af23a06553254d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9"
I20260807 22:43:51.527945 28582 raft_consensus.cc:3037] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:51.529969 28430 raft_consensus.cc:2454] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 1.
I20260807 22:43:51.530545 28204 leader_election.cc:302] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81ee3502329748c1af23a06553254d6e, cfa9a9ee55574cbc8958c35681cca9ad; no voters:
I20260807 22:43:51.530903 28759 raft_consensus.cc:2781] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:51.531504 28759 raft_consensus.cc:686] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 1 LEADER]: Becoming Leader. State: Replica: 81ee3502329748c1af23a06553254d6e, State: Running, Role: LEADER
I20260807 22:43:51.531602 28582 raft_consensus.cc:2454] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ee3502329748c1af23a06553254d6e in term 1.
I20260807 22:43:51.532161 28759 consensus_queue.cc:237] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } }
I20260807 22:43:51.533000 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.532383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:51053)
0807 22:43:51.532674 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.532801 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 22:43:51.536146 28126 catalog_manager.cc:5495] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 81ee3502329748c1af23a06553254d6e (127.20.10.129). New cstate: current_term: 1 leader_uuid: "81ee3502329748c1af23a06553254d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ee3502329748c1af23a06553254d6e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:51.561013 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.554788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.20.10.129:41251)
0807 22:43:51.555054 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.555060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.555082 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:43:51.555397 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.555403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.555494 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.555604 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:51.556018 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.556819 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.558965 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.559435 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.559593 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.560069 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:51.560303 (+ 234us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:51.560421 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.560596 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.560695 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:43:51.565217 28430 raft_consensus.cc:1261] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Refusing update from remote peer 81ee3502329748c1af23a06553254d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:51.565210 28582 raft_consensus.cc:1261] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Refusing update from remote peer 81ee3502329748c1af23a06553254d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:51.566030 28760 consensus_queue.cc:1035] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfa9a9ee55574cbc8958c35681cca9ad" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:51.566496 28759 consensus_queue.cc:1035] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fe4e3dfdec4bc6ad353e805e0cf8f9" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:51.569252 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.568696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:57579)
0807 22:43:51.568888 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.569008 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:43:51.569586 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.568964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:41493)
0807 22:43:51.569317 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.569416 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260807 22:43:51.576957 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.576222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.130:59021)
0807 22:43:51.576575 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.576691 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260807 22:43:51.578028 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.576899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:40439)
0807 22:43:51.577698 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.577818 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":654}
I20260807 22:43:51.578625 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.577913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.131:49391)
0807 22:43:51.578261 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.578381 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260807 22:43:51.590691 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.589998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.20.10.129:35707)
0807 22:43:51.590268 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.590397 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 22:43:51.592813 28126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:51.593104 28126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:51.602605 28126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d231b5f21b2e409599df65ad0ca5d224: Sending DeleteTablet for 3 replicas of tablet f50fbc054be4427397531b7a82e500a0
I20260807 22:43:51.603960 28249 tablet_service.cc:1514] Processing DeleteTablet for tablet f50fbc054be4427397531b7a82e500a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:51 UTC) from {username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:36702
I20260807 22:43:51.604221 28405 tablet_service.cc:1514] Processing DeleteTablet for tablet f50fbc054be4427397531b7a82e500a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:51 UTC) from {username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:55480
I20260807 22:43:51.604575 28560 tablet_service.cc:1514] Processing DeleteTablet for tablet f50fbc054be4427397531b7a82e500a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:51 UTC) from {username='oryx', principal='oryx/127.20.10.188@KRBTEST.COM'} at 127.0.0.1:54486
I20260807 22:43:51.604887 28773 tablet_replica.cc:330] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: stopping tablet replica
I20260807 22:43:51.605154 28773 raft_consensus.cc:2229] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:51.605693 28773 raft_consensus.cc:2258] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:51.606005 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 28090
I20260807 22:43:51.607203 28773 ts_tablet_manager.cc:1905] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:51.615353 28775 tablet_replica.cc:330] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: stopping tablet replica
I20260807 22:43:51.615778 28773 ts_tablet_manager.cc:1918] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:51.615999 28773 log.cc:1196] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/f50fbc054be4427397531b7a82e500a0
I20260807 22:43:51.616027 28775 raft_consensus.cc:2229] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:51.616578 28773 ts_tablet_manager.cc:1939] T f50fbc054be4427397531b7a82e500a0 P 81ee3502329748c1af23a06553254d6e: Deleting consensus metadata
I20260807 22:43:51.616642 28775 raft_consensus.cc:2258] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:51.618220 28112 catalog_manager.cc:4841] TS 81ee3502329748c1af23a06553254d6e (127.20.10.129:41251): tablet f50fbc054be4427397531b7a82e500a0 (table test-table [id=0f2d4787ec184678957131d7c28a5ceb]) successfully deleted
I20260807 22:43:51.618659 28775 ts_tablet_manager.cc:1905] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:51.623293 28774 tablet_replica.cc:330] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: stopping tablet replica
I20260807 22:43:51.623554 28774 raft_consensus.cc:2229] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:51.623970 28774 raft_consensus.cc:2258] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:51.625628 28774 ts_tablet_manager.cc:1905] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:51.626523 28775 ts_tablet_manager.cc:1918] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:51.626708 28775 log.cc:1196] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/f50fbc054be4427397531b7a82e500a0
I20260807 22:43:51.627198 28775 ts_tablet_manager.cc:1939] T f50fbc054be4427397531b7a82e500a0 P 93fe4e3dfdec4bc6ad353e805e0cf8f9: Deleting consensus metadata
I20260807 22:43:51.628268 28114 catalog_manager.cc:4841] TS 93fe4e3dfdec4bc6ad353e805e0cf8f9 (127.20.10.131:35225): tablet f50fbc054be4427397531b7a82e500a0 (table test-table [id=0f2d4787ec184678957131d7c28a5ceb]) successfully deleted
I20260807 22:43:51.629393 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.608248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.20.10.129:41251)
0807 22:43:51.608635 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.608645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.608911 (+ 266us) server_negotiation.cc:407] Connection header received
0807 22:43:51.609216 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.609223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.609324 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.609411 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:51.611735 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.613085 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.615328 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.616096 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.616254 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.620500 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:51.620520 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:51.620538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:51.620587 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:51.624997 (+ 4410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.626207 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.626222 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.626226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.626440 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.627007 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.627013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.627017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.627579 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:51.627929 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.628650 (+ 721us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.628948 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260807 22:43:51.631340 28313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:51.632946 28774 ts_tablet_manager.cc:1918] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:51.633113 28774 log.cc:1196] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/f50fbc054be4427397531b7a82e500a0
I20260807 22:43:51.633643 28774 ts_tablet_manager.cc:1939] T f50fbc054be4427397531b7a82e500a0 P cfa9a9ee55574cbc8958c35681cca9ad: Deleting consensus metadata
I20260807 22:43:51.635089 28114 catalog_manager.cc:4841] TS cfa9a9ee55574cbc8958c35681cca9ad (127.20.10.130:38223): tablet f50fbc054be4427397531b7a82e500a0 (table test-table [id=0f2d4787ec184678957131d7c28a5ceb]) successfully deleted
I20260807 22:43:51.769678 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 28090
I20260807 22:43:51.803188 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 28337
I20260807 22:43:51.824640 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.804117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.20.10.130:38223)
0807 22:43:51.804334 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.804338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.804442 (+ 104us) server_negotiation.cc:407] Connection header received
0807 22:43:51.804595 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.804599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.804668 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.804768 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:51.806233 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.807033 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.808610 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.809072 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.809177 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.817080 (+ 7903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:51.817101 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:51.817119 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:51.817167 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:51.821314 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.822341 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.822350 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.822352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.822513 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.823102 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.823106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.823108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.823489 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:51.823722 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.824031 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.824229 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:43:51.825626 28475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:51.904639 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.904094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58542)
0807 22:43:51.904286 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.904436 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 22:43:51.906122 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.905586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:43009 (local address 127.0.0.1:58554)
0807 22:43:51.905800 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:51.905925 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:43009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 22:43:51.935922 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 28337
I20260807 22:43:51.962306 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 28494
I20260807 22:43:51.980124 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:51.963182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.20.10.131:35225)
0807 22:43:51.963827 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:51.963832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:51.963852 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:51.963913 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:51.963917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:51.963997 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:51.964112 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:51.965910 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.966874 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.968434 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:51.968967 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:51.969089 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:51.972488 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:51.972507 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:51.972525 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:51.972566 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:51.976527 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.977523 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.977532 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.977534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.977746 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:51.978385 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:51.978390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:51.978393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:51.978962 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:51.979216 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:51.979600 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 22:43:51.979805 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":410,"threads_started":1}
I20260807 22:43:51.981074 28624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:52.085525 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 28494
I20260807 22:43:52.109454 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27759
I20260807 22:43:52.110909 27948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:52.241600 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27759
I20260807 22:43:52.270814 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 27874
I20260807 22:43:52.272063 28161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:52.409755 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 27874
2026-08-07T22:43:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (13169 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:52 dist-test-slave-csg5 krb5kdc[28788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:52 dist-test-slave-csg5 krb5kdc[28788](info): set up 2 sockets
Aug 07 22:43:52 dist-test-slave-csg5 krb5kdc[28788](info): commencing operation
krb5kdc: starting...
W20260807 22:43:53.659427 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.005s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:43:53 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab.
Entry for principal oryx/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab.
I20260807 22:43:53.972225 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35027
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab
--principal=oryx/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:54.344869 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:54.345237 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:54.345345 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:54.357234 28802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:54.357412 28802 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:43:54.357555 28802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:54.357684 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:54.357788 28802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:54.357883 28802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:54.369727 28802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35027
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.190.keytab
--principal=oryx/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:54.371270 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:54.372768 28802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:54.387115 28811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:54.387071 28809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:54.387140 28808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:54.388099 28802 server_base.cc:1034] running on GCE node
I20260807 22:43:54.389081 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:54.390880 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:54.392172 28802 hybrid_clock.cc:648] HybridClock initialized: now 1786142634392084 us; error 61 us; skew 500 ppm
I20260807 22:43:54.392568 28802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:54 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:54.396262 28802 init.cc:377] Logged in from keytab as oryx/127.20.10.190@KRBTEST.COM (short username oryx)
I20260807 22:43:54.398137 28802 webserver.cc:460] Webserver started at http://127.20.10.190:36855/ using document root <none> and password file <none>
I20260807 22:43:54.399017 28802 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:54.399115 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:54.399415 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:54.402736 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6c7a79c953e946eb8b458c3ccd730856"
format_stamp: "Formatted at 2026-08-07 22:43:54 on dist-test-slave-csg5"
server_key: "3d3fce91116f5d03e07a552b1ad794e3"
server_key_iv: "99f0ddb50e99a3ae68d0698566901404"
server_key_version: "encryptionkey@0"
I20260807 22:43:54.403586 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6c7a79c953e946eb8b458c3ccd730856"
format_stamp: "Formatted at 2026-08-07 22:43:54 on dist-test-slave-csg5"
server_key: "3d3fce91116f5d03e07a552b1ad794e3"
server_key_iv: "99f0ddb50e99a3ae68d0698566901404"
server_key_version: "encryptionkey@0"
I20260807 22:43:54.409461 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:43:54.413614 28818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:54.415244 28802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 22:43:54.415516 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6c7a79c953e946eb8b458c3ccd730856"
format_stamp: "Formatted at 2026-08-07 22:43:54 on dist-test-slave-csg5"
server_key: "3d3fce91116f5d03e07a552b1ad794e3"
server_key_iv: "99f0ddb50e99a3ae68d0698566901404"
server_key_version: "encryptionkey@0"
I20260807 22:43:54.415727 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:54.445832 28802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:54.451602 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:54.451910 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:54.479794 28802 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:35027
I20260807 22:43:54.479897 28883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:35027 every 8 connection(s)
I20260807 22:43:54.481441 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:54.484622 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 28802
I20260807 22:43:54.484897 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:43:54.485766 20522 external_mini_cluster.cc:1428] Setting key 1715e4bb3b457729ca507f0130fdbec9
I20260807 22:43:54.486526 28885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:54.503840 28885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Bootstrap starting.
Aug 07 22:43:54 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:54.508642 28885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:54.510243 28885 log.cc:824] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:54.513667 28885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: No bootstrap required, opened a new log
I20260807 22:43:54.515200 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:54.488308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.20.10.190:35027)
0807 22:43:54.489315 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:54.489330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:54.489374 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:43:54.492285 (+ 2911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:54.492314 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:54.492843 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:54.493262 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:54.495430 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:54.496893 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:54.498843 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:54.499697 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:54.499922 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:54.505366 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:54.505381 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:54.505410 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:54.505486 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:54.510311 (+ 4825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:54.511329 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:54.511337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:54.511344 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:54.511557 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:54.512072 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:54.512076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:54.512078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:54.512691 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:54.513047 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:54.513514 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 22:43:54.514034 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":441,"threads_started":1}
I20260807 22:43:54.523216 28885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } }
I20260807 22:43:54.523674 28885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:54.523787 28885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7a79c953e946eb8b458c3ccd730856, State: Initialized, Role: FOLLOWER
I20260807 22:43:54.524394 28885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } }
I20260807 22:43:54.524654 28885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:54.524823 28885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:54.525010 28885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:54.527487 28885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } }
I20260807 22:43:54.528115 28885 leader_election.cc:302] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c7a79c953e946eb8b458c3ccd730856; no voters:
I20260807 22:43:54.528846 28885 leader_election.cc:288] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:54.529026 28890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:54.530565 28890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [term 1 LEADER]: Becoming Leader. State: Replica: 6c7a79c953e946eb8b458c3ccd730856, State: Running, Role: LEADER
I20260807 22:43:54.531325 28885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:54.531243 28890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } }
I20260807 22:43:54.534682 28891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c7a79c953e946eb8b458c3ccd730856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } } }
I20260807 22:43:54.534814 28892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c7a79c953e946eb8b458c3ccd730856. Latest consensus state: current_term: 1 leader_uuid: "6c7a79c953e946eb8b458c3ccd730856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7a79c953e946eb8b458c3ccd730856" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35027 } } }
I20260807 22:43:54.535166 28891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:54.535264 28892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:54.538508 28900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:54.543861 28900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:54.553751 28900 catalog_manager.cc:1347] Generated new cluster ID: d03af6a650c94ae2969347f1738eb23f
I20260807 22:43:54.553920 28900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:54.570348 28900 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:54.572037 28900 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:54.585036 28900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Generated new TSK 0
I20260807 22:43:54.586359 28900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
I20260807 22:43:54.672072 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35027
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:55.073988 28913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:55.074375 28913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:55.074554 28913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:55.082502 28913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:55.082718 28913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:55.082893 28913 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:55.083040 28913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:43:55.094585 28913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:55.096109 28913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:55.097546 28913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:55.113148 28918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:55.113010 28921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:55.113108 28919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:55.114928 28913 server_base.cc:1034] running on GCE node
I20260807 22:43:55.115671 28913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:55.116739 28913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:55.117997 28913 hybrid_clock.cc:648] HybridClock initialized: now 1786142635117927 us; error 41 us; skew 500 ppm
I20260807 22:43:55.118299 28913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:55 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142635, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:55.122531 28913 init.cc:377] Logged in from keytab as oryx/127.20.10.129@KRBTEST.COM (short username oryx)
I20260807 22:43:55.124336 28913 webserver.cc:460] Webserver started at http://127.20.10.129:33413/ using document root <none> and password file <none>
I20260807 22:43:55.125200 28913 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:55.125303 28913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:55.125666 28913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:55.129184 28913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "076d85cfb19943789f6075bb46ee4f39"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "1f0dcda3d45f49461cebfc94e14b3dc4"
server_key_iv: "45f6439eb95491e0d66a99cc724761dc"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.130223 28913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "076d85cfb19943789f6075bb46ee4f39"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "1f0dcda3d45f49461cebfc94e14b3dc4"
server_key_iv: "45f6439eb95491e0d66a99cc724761dc"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.136798 28913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 22:43:55.141150 28928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:55.143129 28913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:43:55.143349 28913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "076d85cfb19943789f6075bb46ee4f39"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "1f0dcda3d45f49461cebfc94e14b3dc4"
server_key_iv: "45f6439eb95491e0d66a99cc724761dc"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.143615 28913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:55.181690 28913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:55.187160 28913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:55.187491 28913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:55.190891 28913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:55.190984 28913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:55.191134 28913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:55.191227 28913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:55 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142635, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:55.222512 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.191947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48627 (local address 127.20.10.190:35027)
0807 22:43:55.192543 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:55.192552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:55.193962 (+ 1410us) server_negotiation.cc:407] Connection header received
0807 22:43:55.197485 (+ 3523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:55.197492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:55.197619 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:55.197786 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:55.200820 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.202175 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.205182 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.205678 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.205856 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.212174 (+ 6318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:55.212186 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:55.212191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:55.212259 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:55.217499 (+ 5240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.218939 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.218945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.218947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.219126 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.219952 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.219958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.219962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.220417 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:55.220579 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:55.221902 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0807 22:43:55.222139 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":308,"threads_started":1}
I20260807 22:43:55.224658 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.192737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.129:48627)
0807 22:43:55.193725 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:55.193783 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:55.196958 (+ 3175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:55.198109 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:55.198119 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:55.199420 (+ 1301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:55.200510 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.200567 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.202531 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.202540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.204909 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.204932 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.206014 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.206024 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.206320 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.207396 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:55.207444 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:55.211679 (+ 4235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:55.217847 (+ 6168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.217864 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.217886 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.218695 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.219285 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.219294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.219301 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.219770 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.220581 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:55.220595 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:55.221317 (+ 722us) client_negotiation.cc:769] Sending connection context
0807 22:43:55.221742 (+ 425us) client_negotiation.cc:240] Negotiation successful
0807 22:43:55.222557 (+ 815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":395,"threads_started":1}
I20260807 22:43:55.253605 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.234151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:42683 (local address 127.20.10.190:35027)
0807 22:43:55.234380 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:55.234389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:55.235600 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 22:43:55.235888 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:55.235898 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:55.236017 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:55.236170 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:55.237813 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.238798 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.241865 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.242614 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.242851 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.246053 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:55.246059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:55.246063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:55.246138 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:55.249802 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.251110 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.251119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.251123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.251378 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.251852 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.251858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.251862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.252205 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:55.252393 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:55.252907 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 22:43:55.253191 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:43:55.253738 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.234238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.129:42683)
0807 22:43:55.235436 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:55.235460 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:55.235702 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:55.236214 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:55.236220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:55.237161 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:55.237613 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.237636 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.238964 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.238971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.241525 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.241558 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.242943 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.242950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.243131 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.244099 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:55.244134 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:55.245742 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:55.250012 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.250022 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.250029 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.250893 (+ 864us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.251544 (+ 651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.251550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.251554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.251721 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.252360 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:55.252365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:55.252593 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 22:43:55.252768 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 22:43:55.253181 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"thread_start_us":674,"threads_started":1}
I20260807 22:43:55.260535 28913 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:46785
I20260807 22:43:55.260576 29060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:46785 every 8 connection(s)
I20260807 22:43:55.261947 28913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:55.264617 29061 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35027
I20260807 22:43:55.264900 29061 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:55.265745 29061 heartbeater.cc:511] Master 127.20.10.190:35027 requested a full tablet report, sending...
I20260807 22:43:55.268476 28836 ts_manager.cc:194] Registered new tserver with Master: 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785)
I20260807 22:43:55.268567 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 28913
I20260807 22:43:55.268859 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:55.269557 20522 external_mini_cluster.cc:1428] Setting key 3527e789fe75636c36c1d6becb6117ee
I20260807 22:43:55.270462 28836 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:48627
WARNING: no policy specified for oryx/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
I20260807 22:43:55.343961 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35027
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:55.708586 29065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:55.708904 29065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:55.709007 29065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:55.716652 29065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:55.716800 29065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:55.716889 29065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:55.716975 29065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:43:55.728657 29065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:55.730144 29065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:55.731894 29065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:55.747860 29073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:55.747828 29071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:55.747874 29070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:55.752683 29065 server_base.cc:1034] running on GCE node
I20260807 22:43:55.753473 29065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:55.754632 29065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:55.755977 29065 hybrid_clock.cc:648] HybridClock initialized: now 1786142635755873 us; error 70 us; skew 500 ppm
I20260807 22:43:55.756367 29065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:55 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142635, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:55.760412 29065 init.cc:377] Logged in from keytab as oryx/127.20.10.130@KRBTEST.COM (short username oryx)
I20260807 22:43:55.762434 29065 webserver.cc:460] Webserver started at http://127.20.10.130:41965/ using document root <none> and password file <none>
I20260807 22:43:55.763392 29065 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:55.763551 29065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:55.764014 29065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:55.767737 29065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "de19e6fea21d4611a6e92c99d84cef5c"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "c30932052642cbe41ecfcbac2b9328d5"
server_key_iv: "33cf26961026a2388841989f3d4565ed"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.768741 29065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "de19e6fea21d4611a6e92c99d84cef5c"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "c30932052642cbe41ecfcbac2b9328d5"
server_key_iv: "33cf26961026a2388841989f3d4565ed"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.775900 29065 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 22:43:55.780586 29080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:55.782326 29065 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:43:55.782554 29065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "de19e6fea21d4611a6e92c99d84cef5c"
format_stamp: "Formatted at 2026-08-07 22:43:55 on dist-test-slave-csg5"
server_key: "c30932052642cbe41ecfcbac2b9328d5"
server_key_iv: "33cf26961026a2388841989f3d4565ed"
server_key_version: "encryptionkey@0"
I20260807 22:43:55.782960 29065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:55.805671 29065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:55.810891 29065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:55.811188 29065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:55.814448 29065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:55.814586 29065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:55.814839 29065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:55.814927 29065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:55 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142635, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:55.845280 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.815020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:54209 (local address 127.20.10.190:35027)
0807 22:43:55.815693 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:55.815703 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:55.817145 (+ 1442us) server_negotiation.cc:407] Connection header received
0807 22:43:55.819765 (+ 2620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:55.819771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:55.819891 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:55.820048 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:55.823001 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.824302 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.826884 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.827823 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.828003 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.835604 (+ 7601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:55.835616 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:55.835623 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:55.835690 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:55.839917 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.841544 (+ 1627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.841550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.841554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.841775 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.842705 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.842714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.842719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.843076 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:55.843339 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:55.844465 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0807 22:43:55.844767 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":375,"threads_started":1}
I20260807 22:43:55.846546 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.815946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.130:54209)
0807 22:43:55.816933 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:55.816988 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:55.819377 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:55.820369 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:55.820383 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:55.821458 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:55.822735 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.822797 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.824677 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.824682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.826566 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.826582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.828263 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.828273 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.828689 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.830043 (+ 1354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:55.830100 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:55.835139 (+ 5039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:55.840206 (+ 5067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.840225 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.840250 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.841285 (+ 1035us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.841977 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.841986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.841996 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.842482 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.843449 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:55.843462 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:55.843990 (+ 528us) client_negotiation.cc:769] Sending connection context
0807 22:43:55.844309 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 22:43:55.845007 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":428,"threads_started":1}
I20260807 22:43:55.874359 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.854978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49357 (local address 127.20.10.190:35027)
0807 22:43:55.855312 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:55.855321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:55.856456 (+ 1135us) server_negotiation.cc:407] Connection header received
0807 22:43:55.856714 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:55.856722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:55.856812 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:55.856933 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:55.858157 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.859085 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.861600 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.862086 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.862281 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.865698 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:55.865710 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:55.865719 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:55.865788 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:55.870869 (+ 5081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.871805 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.871814 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.871818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.872017 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:55.872614 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:55.872621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:55.872625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:55.873013 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:55.873214 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:55.873657 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 22:43:55.873944 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:43:55.874473 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:55.854771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.130:49357)
0807 22:43:55.856318 (+ 1547us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:55.856339 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:55.856556 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:55.856969 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:55.856974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:55.857608 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:55.857945 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.857956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.859325 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.859334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.861397 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:55.861413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:55.862332 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:55.862339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:55.862546 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:55.863442 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:55.863494 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:55.865280 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:55.871040 (+ 5760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.871046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.871049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.871575 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.872213 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:55.872224 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:55.872227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:55.872404 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:55.873162 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:55.873168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:55.873388 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 22:43:55.873550 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 22:43:55.873960 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1362,"thread_start_us":1120,"threads_started":1}
I20260807 22:43:55.887292 29065 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:46621
I20260807 22:43:55.887398 29207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:46621 every 8 connection(s)
I20260807 22:43:55.888914 29065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:55.891841 29208 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35027
I20260807 22:43:55.892199 29208 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:55.893060 29208 heartbeater.cc:511] Master 127.20.10.190:35027 requested a full tablet report, sending...
I20260807 22:43:55.894935 28835 ts_manager.cc:194] Registered new tserver with Master: de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:55.895995 28835 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:54209
I20260807 22:43:55.896410 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29065
I20260807 22:43:55.896721 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:55.897259 20522 external_mini_cluster.cc:1428] Setting key e923182f0c68e1ce34e5e18601b902ff
WARNING: no policy specified for oryx/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
I20260807 22:43:55.967164 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35027
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:56.274889 29061 heartbeater.cc:503] Master 127.20.10.190:35027 was elected leader, sending a full tablet report...
W20260807 22:43:56.368151 29213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:56.368479 29213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:56.368592 29213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:56.376777 29213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:56.376915 29213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:56.377013 29213 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:56.377099 29213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:43:56.388163 29213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.10.190:35027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:43:56.389667 29213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:56.391175 29213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:56.407287 29218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:56.407334 29219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:56.407356 29221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:56.408268 29213 server_base.cc:1034] running on GCE node
I20260807 22:43:56.408999 29213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:56.410137 29213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:56.411443 29213 hybrid_clock.cc:648] HybridClock initialized: now 1786142636411343 us; error 64 us; skew 500 ppm
I20260807 22:43:56.411983 29213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:43:56 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142636, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:56.416080 29213 init.cc:377] Logged in from keytab as oryx/127.20.10.131@KRBTEST.COM (short username oryx)
I20260807 22:43:56.418068 29213 webserver.cc:460] Webserver started at http://127.20.10.131:36451/ using document root <none> and password file <none>
I20260807 22:43:56.418967 29213 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:56.419076 29213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:56.419467 29213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:56.423569 29213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "14508720f7164ac59256359e0e134f48"
format_stamp: "Formatted at 2026-08-07 22:43:56 on dist-test-slave-csg5"
server_key: "c0d2a26f6a75219793f3e5d001f756c4"
server_key_iv: "34a64eed8483ee1a728da7f49cc05c44"
server_key_version: "encryptionkey@0"
I20260807 22:43:56.424578 29213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "14508720f7164ac59256359e0e134f48"
format_stamp: "Formatted at 2026-08-07 22:43:56 on dist-test-slave-csg5"
server_key: "c0d2a26f6a75219793f3e5d001f756c4"
server_key_iv: "34a64eed8483ee1a728da7f49cc05c44"
server_key_version: "encryptionkey@0"
I20260807 22:43:56.431905 29213 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 22:43:56.437317 29229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:56.439502 29213 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:43:56.439766 29213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "14508720f7164ac59256359e0e134f48"
format_stamp: "Formatted at 2026-08-07 22:43:56 on dist-test-slave-csg5"
server_key: "c0d2a26f6a75219793f3e5d001f756c4"
server_key_iv: "34a64eed8483ee1a728da7f49cc05c44"
server_key_version: "encryptionkey@0"
I20260807 22:43:56.440022 29213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:56.458563 29213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:56.464059 29213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:56.464421 29213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:56.468230 29213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:56.468376 29213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:56.468590 29213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:56.468684 29213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:56 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142636, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:56.498749 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:56.468817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:56343 (local address 127.20.10.190:35027)
0807 22:43:56.469836 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:56.469844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:56.471043 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 22:43:56.473512 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:56.473521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:56.473656 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:56.473853 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:56.476917 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.478214 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.481502 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.482627 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.482829 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:56.488870 (+ 6041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:56.488879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:56.488884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:56.488935 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:56.493585 (+ 4650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:56.495131 (+ 1546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:56.495144 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:56.495146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:56.495410 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:56.496317 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:56.496325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:56.496332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:56.496791 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:56.496999 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:56.497943 (+ 944us) server_negotiation.cc:299] Negotiation successful
0807 22:43:56.498262 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":738,"threads_started":1}
I20260807 22:43:56.500599 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:56.469728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.131:56343)
0807 22:43:56.470814 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:56.470880 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:56.473128 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:56.474346 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:56.474356 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:56.475566 (+ 1210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:56.476643 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:56.476691 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.478408 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.478416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:56.481249 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:56.481270 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.482939 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.482946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:56.483275 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:56.484270 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:56.484306 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:56.488500 (+ 4194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:56.493885 (+ 5385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:56.493901 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:56.493932 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:56.494867 (+ 935us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:56.495596 (+ 729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:56.495610 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:56.495618 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:56.496116 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:56.496985 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:56.496994 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:56.497487 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 22:43:56.497815 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 22:43:56.498465 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":461,"threads_started":1}
I20260807 22:43:56.525743 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:56.509472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:60213 (local address 127.20.10.190:35027)
0807 22:43:56.509678 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:56.509683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:56.510547 (+ 864us) server_negotiation.cc:407] Connection header received
0807 22:43:56.510808 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:56.510814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:56.510900 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:56.511017 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:56.512067 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.512827 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.514186 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.514618 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.514780 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:56.517953 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:56.517959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:56.517963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:56.518002 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:56.522132 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:56.523466 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:56.523480 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:56.523484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:56.523628 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:56.524053 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:56.524058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:56.524064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:56.524392 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:56.524569 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:56.525020 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 22:43:56.525315 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:43:56.526062 29320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:56.509542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.20.10.131:60213)
0807 22:43:56.510398 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:56.510420 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:56.510632 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:56.511055 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:56.511059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:56.511589 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:56.511904 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:56.511913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.512979 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.512985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:56.514033 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:56.514045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.514873 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.514880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:56.515077 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:56.516032 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:56.516053 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:56.517687 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:56.522439 (+ 4752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:56.522444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:56.522447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:56.523276 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:56.523772 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:56.523778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:56.523782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:56.523924 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:56.524527 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:56.524532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:56.524741 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 22:43:56.524910 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 22:43:56.525354 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":440,"threads_started":1}
I20260807 22:43:56.534147 29213 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:40313
I20260807 22:43:56.534232 29353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:40313 every 8 connection(s)
I20260807 22:43:56.535569 29213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:56.538297 29354 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35027
I20260807 22:43:56.538538 29354 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:56.539352 29354 heartbeater.cc:511] Master 127.20.10.190:35027 requested a full tablet report, sending...
I20260807 22:43:56.540927 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29213
I20260807 22:43:56.541164 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:56.541193 28835 ts_manager.cc:194] Registered new tserver with Master: 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313)
I20260807 22:43:56.541708 20522 external_mini_cluster.cc:1428] Setting key eaf88845405f0bbdb9d9cffa2bdd7cee
I20260807 22:43:56.542307 28835 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:56343
I20260807 22:43:56.546059 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:43:56 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142636, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:43:56 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:56 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:56.899107 29208 heartbeater.cc:503] Master 127.20.10.190:35027 was elected leader, sending a full tablet report...
I20260807 22:43:57.545747 29354 heartbeater.cc:503] Master 127.20.10.190:35027 was elected leader, sending a full tablet report...
W20260807 22:43:57.745533 29363 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.133s user 0.004s sys 0.017s
W20260807 22:43:57.747073 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:56.605481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.20.10.190:35027)
0807 22:43:56.605688 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:56.605694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:56.605745 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:43:56.605922 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:56.605927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:56.606044 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:56.606224 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:56.607759 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.608648 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.610115 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:56.610727 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:56.610848 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.746716 (+1135868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54008: BlockingRecv error: recv got EOF from 127.0.0.1:54008 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 22:43:57 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142636, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:57.774798 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.752566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.20.10.190:35027)
0807 22:43:57.752801 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:57.752806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:57.753277 (+ 471us) server_negotiation.cc:407] Connection header received
0807 22:43:57.753482 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:57.753487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:57.753567 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:57.753679 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:57.755555 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.756516 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.758053 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.758798 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.758941 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.768067 (+ 9126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:57.768074 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:57.768078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:57.768232 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:57.771469 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:57.772466 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:57.772471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:57.772474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:57.772627 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:57.773258 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:57.773262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:57.773264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:57.773589 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:57.773927 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:57.774203 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:43:57.774445 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:43:57.780018 28835 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:57.787953 28835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:57.821641 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.803234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46785 (local address 127.0.0.1:56172)
0807 22:43:57.805902 (+ 2668us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:57.805968 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:57.806220 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:57.808339 (+ 2119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:57.808352 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:57.808402 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:57.809021 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.809044 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.812772 (+ 3728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.812784 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.816631 (+ 3847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.816656 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.820385 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.820393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.820663 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.820738 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 22:43:57.820888 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:43:57.821160 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2314,"thread_start_us":13977,"threads_started":1}
I20260807 22:43:57.821797 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.803402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.20.10.129:46785)
0807 22:43:57.807077 (+ 3675us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:57.807092 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:57.807135 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:43:57.807284 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:57.807298 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:57.807655 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:57.807918 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:57.809372 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.811799 (+ 2427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.818045 (+ 6246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.820058 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.820269 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.820541 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:57.821226 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 22:43:57.821456 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3421,"thread_start_us":3256,"threads_started":1}
I20260807 22:43:57.826730 28984 tablet_service.cc:1467] Processing CreateTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 (DEFAULT_TABLE table=test-table [id=918eeed0b0b04a6daebbc0e9a9922570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:57.827574 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.804149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.20.10.130:46621)
0807 22:43:57.812612 (+ 8463us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:57.812627 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:57.816829 (+ 4202us) server_negotiation.cc:407] Connection header received
0807 22:43:57.816980 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:57.816989 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:57.817350 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:57.817616 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:57.818323 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.820978 (+ 2655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.824272 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.825983 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.826142 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.826398 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:57.826989 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 22:43:57.827177 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8188,"thread_start_us":7993,"threads_started":1}
I20260807 22:43:57.827615 29379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.803474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:46621 (local address 127.0.0.1:47482)
0807 22:43:57.816492 (+ 13018us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:57.816575 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:57.816745 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:57.817582 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:57.817588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:57.817615 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:57.818090 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.818107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.821134 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.821141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.824025 (+ 2884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.824044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.826442 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.826449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.826770 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.826808 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:43:57.826929 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:43:57.827089 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12839,"mutex_wait_us":139,"thread_start_us":1170,"threads_started":1}
I20260807 22:43:57.827661 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.805643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:40313 (local address 127.0.0.1:37428)
0807 22:43:57.818181 (+ 12538us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:57.818203 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:57.818383 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:57.819133 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:57.819139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:57.819169 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:57.819579 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.819647 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.821731 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.821737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.824337 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:57.824348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.826610 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.826618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:57.826866 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.826906 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:43:57.827010 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 22:43:57.827182 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11794,"mutex_wait_us":28,"thread_start_us":11193,"threads_started":1}
I20260807 22:43:57.828514 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:57.806066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.20.10.131:40313)
0807 22:43:57.818045 (+ 11979us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:57.818052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:57.818381 (+ 329us) server_negotiation.cc:407] Connection header received
0807 22:43:57.818561 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:57.818570 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:57.818884 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:57.819181 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:57.819861 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.821523 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.824516 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:57.826260 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:57.826431 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:57.826726 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:57.827925 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0807 22:43:57.828133 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11796,"thread_start_us":11645,"threads_started":1}
I20260807 22:43:57.828879 28984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55586542f934e3eabbbed9c6a9d0d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:57.832301 29281 tablet_service.cc:1467] Processing CreateTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 (DEFAULT_TABLE table=test-table [id=918eeed0b0b04a6daebbc0e9a9922570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:57.832409 29132 tablet_service.cc:1467] Processing CreateTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 (DEFAULT_TABLE table=test-table [id=918eeed0b0b04a6daebbc0e9a9922570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:57.833771 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55586542f934e3eabbbed9c6a9d0d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:57.833909 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55586542f934e3eabbbed9c6a9d0d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:57.847182 29381 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Bootstrap starting.
I20260807 22:43:57.851677 29383 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Bootstrap starting.
I20260807 22:43:57.853821 29381 tablet_bootstrap.cc:654] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:57.854148 29382 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Bootstrap starting.
I20260807 22:43:57.855371 29383 tablet_bootstrap.cc:654] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:57.855472 29381 log.cc:824] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:57.856909 29383 log.cc:824] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:57.858358 29382 tablet_bootstrap.cc:654] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:57.859853 29382 log.cc:824] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:57.860061 29381 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: No bootstrap required, opened a new log
I20260807 22:43:57.860424 29381 ts_tablet_manager.cc:1397] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260807 22:43:57.861619 29383 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: No bootstrap required, opened a new log
I20260807 22:43:57.861932 29383 ts_tablet_manager.cc:1397] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 22:43:57.870059 29382 tablet_bootstrap.cc:492] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: No bootstrap required, opened a new log
I20260807 22:43:57.870417 29382 ts_tablet_manager.cc:1397] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260807 22:43:57.870597 29383 raft_consensus.cc:348] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.871110 29383 raft_consensus.cc:374] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:57.871268 29383 raft_consensus.cc:729] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de19e6fea21d4611a6e92c99d84cef5c, State: Initialized, Role: FOLLOWER
I20260807 22:43:57.871990 29383 consensus_queue.cc:260] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.874176 29381 raft_consensus.cc:348] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.874744 29381 raft_consensus.cc:374] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:57.874866 29381 raft_consensus.cc:729] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Initialized, Role: FOLLOWER
I20260807 22:43:57.875784 29381 consensus_queue.cc:260] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.877234 29383 ts_tablet_manager.cc:1428] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260807 22:43:57.879262 29381 ts_tablet_manager.cc:1428] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Time spent starting tablet: real 0.019s user 0.015s sys 0.005s
I20260807 22:43:57.883857 29382 raft_consensus.cc:348] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.884351 29382 raft_consensus.cc:374] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:57.884487 29382 raft_consensus.cc:729] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14508720f7164ac59256359e0e134f48, State: Initialized, Role: FOLLOWER
I20260807 22:43:57.885134 29382 consensus_queue.cc:260] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:57.887090 29382 ts_tablet_manager.cc:1428] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Time spent starting tablet: real 0.017s user 0.007s sys 0.012s
W20260807 22:43:57.892568 29209 tablet.cc:2369] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:58.016676 29062 tablet.cc:2369] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:58.038504 29355 tablet.cc:2369] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:58.219056 29390 raft_consensus.cc:482] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:58.219549 29390 raft_consensus.cc:504] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.221781 29390 leader_election.cc:288] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:58.230118 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.222256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:32783 (local address 127.20.10.129:46785)
0807 22:43:58.222532 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.222543 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.223326 (+ 783us) server_negotiation.cc:407] Connection header received
0807 22:43:58.223621 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.223626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.223713 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.223833 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:58.224652 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.225456 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.228084 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.229075 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.229299 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.229387 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.229689 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.229824 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:43:58.230111 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.222278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:43729 (local address 127.20.10.130:46621)
0807 22:43:58.222525 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.222547 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.223435 (+ 888us) server_negotiation.cc:407] Connection header received
0807 22:43:58.223671 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.223679 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.223755 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.223870 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:58.224615 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.225456 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.227923 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.228938 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.229153 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.229270 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.229651 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.229805 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:43:58.230296 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.222298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:46621 (local address 127.20.10.131:43729)
0807 22:43:58.223295 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.223311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.223488 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.223887 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.223891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.223917 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:58.224419 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.224432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.225632 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.225638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.227704 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.227722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.229255 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.229263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.229459 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.229486 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.229615 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.229800 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":22,"thread_start_us":574,"threads_started":1}
I20260807 22:43:58.230542 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.222022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46785 (local address 127.20.10.131:32783)
0807 22:43:58.223178 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.223196 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.223425 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.223852 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.223856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.223885 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:58.224462 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.224474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.225636 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.225643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.227873 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.227888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.229352 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.229358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.229519 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.229543 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.229654 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.229805 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":107,"thread_start_us":630,"threads_started":1}
I20260807 22:43:58.231745 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55586542f934e3eabbbed9c6a9d0d30" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c" is_pre_election: true
I20260807 22:43:58.231923 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55586542f934e3eabbbed9c6a9d0d30" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39" is_pre_election: true
I20260807 22:43:58.232420 29159 raft_consensus.cc:2454] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:58.232426 29010 raft_consensus.cc:2454] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:58.233122 29230 leader_election.cc:302] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, 14508720f7164ac59256359e0e134f48, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:58.233608 29390 raft_consensus.cc:2781] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:58.233780 29390 raft_consensus.cc:482] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:58.233937 29390 raft_consensus.cc:3037] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.236450 29390 raft_consensus.cc:504] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.237344 29390 leader_election.cc:288] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Requested vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:58.237921 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55586542f934e3eabbbed9c6a9d0d30" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c"
I20260807 22:43:58.237978 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55586542f934e3eabbbed9c6a9d0d30" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39"
I20260807 22:43:58.238161 29159 raft_consensus.cc:3037] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.238225 29010 raft_consensus.cc:3037] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.240669 29010 raft_consensus.cc:2454] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:58.240672 29159 raft_consensus.cc:2454] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:58.241206 29230 leader_election.cc:302] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14508720f7164ac59256359e0e134f48, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:58.241575 29390 raft_consensus.cc:2781] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:58.242115 29390 raft_consensus.cc:686] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Becoming Leader. State: Replica: 14508720f7164ac59256359e0e134f48, State: Running, Role: LEADER
I20260807 22:43:58.242745 29390 consensus_queue.cc:237] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.247895 28835 catalog_manager.cc:5495] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14508720f7164ac59256359e0e134f48 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "14508720f7164ac59256359e0e134f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } health_report { overall_health: HEALTHY } } }
I20260807 22:43:58.284602 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.277824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.20.10.131:40313)
0807 22:43:58.278127 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.278138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.278410 (+ 272us) server_negotiation.cc:407] Connection header received
0807 22:43:58.278613 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.278617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.278839 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.278991 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:58.279442 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.280254 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.281698 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.282335 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.282458 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.283075 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.283730 (+ 655us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.283889 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.284041 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.284129 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 22:43:58.294955 29010 raft_consensus.cc:1261] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.294957 29159 raft_consensus.cc:1261] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.296231 29393 consensus_queue.cc:1035] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.296608 29390 consensus_queue.cc:1035] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.314424 29397 mvcc.cc:205] Tried to move back new op lower bound from 7316040246442541056 to 7316040246246662144. Current Snapshot: MvccSnapshot[applied={T|T < 7316040246442541056}]
I20260807 22:43:58.316228 29398 mvcc.cc:205] Tried to move back new op lower bound from 7316040246442541056 to 7316040246246662144. Current Snapshot: MvccSnapshot[applied={T|T < 7316040246442541056}]
I20260807 22:43:58.323796 29399 mvcc.cc:205] Tried to move back new op lower bound from 7316040246442541056 to 7316040246246662144. Current Snapshot: MvccSnapshot[applied={T|T < 7316040246442541056}]
I20260807 22:43:58.332671 28836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:58.332983 28836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:58.337193 28836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Sending DeleteTablet for 3 replicas of tablet e55586542f934e3eabbbed9c6a9d0d30
I20260807 22:43:58.338963 28984 tablet_service.cc:1514] Processing DeleteTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56172
I20260807 22:43:58.339259 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47482
I20260807 22:43:58.339602 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet e55586542f934e3eabbbed9c6a9d0d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:37428
I20260807 22:43:58.344527 28836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:58.346547 28836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:58.355754 29411 tablet_replica.cc:330] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: stopping tablet replica
I20260807 22:43:58.356079 29410 tablet_replica.cc:330] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: stopping tablet replica
I20260807 22:43:58.356355 29411 raft_consensus.cc:2229] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:58.356696 29410 raft_consensus.cc:2229] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:58.357152 29411 raft_consensus.cc:2258] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.357371 29410 raft_consensus.cc:2258] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.358223 29409 tablet_replica.cc:330] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: stopping tablet replica
I20260807 22:43:58.358768 29409 raft_consensus.cc:2229] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:58.359334 29409 raft_consensus.cc:2258] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.359759 29411 ts_tablet_manager.cc:1905] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.363521 29409 ts_tablet_manager.cc:1905] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.363951 29410 ts_tablet_manager.cc:1905] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.366468 29281 tablet_service.cc:1467] Processing CreateTablet for tablet 11564109d2d64bd6876739bad9ede908 (DEFAULT_TABLE table=test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:58.366475 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 11564109d2d64bd6876739bad9ede908 (DEFAULT_TABLE table=test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:58.367065 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11564109d2d64bd6876739bad9ede908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.367089 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11564109d2d64bd6876739bad9ede908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.372120 29409 ts_tablet_manager.cc:1918] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:58.372444 29409 log.cc:1196] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/e55586542f934e3eabbbed9c6a9d0d30
I20260807 22:43:58.373147 29409 ts_tablet_manager.cc:1939] T e55586542f934e3eabbbed9c6a9d0d30 P 076d85cfb19943789f6075bb46ee4f39: Deleting consensus metadata
I20260807 22:43:58.375227 29410 ts_tablet_manager.cc:1918] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:58.375443 29410 log.cc:1196] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/e55586542f934e3eabbbed9c6a9d0d30
I20260807 22:43:58.375595 28820 catalog_manager.cc:4841] TS 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785): tablet e55586542f934e3eabbbed9c6a9d0d30 (table test-table [id=918eeed0b0b04a6daebbc0e9a9922570]) successfully deleted
I20260807 22:43:58.375821 29382 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Bootstrap starting.
I20260807 22:43:58.375119 28984 tablet_service.cc:1467] Processing CreateTablet for tablet 11564109d2d64bd6876739bad9ede908 (DEFAULT_TABLE table=test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:58.376397 28984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11564109d2d64bd6876739bad9ede908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.376096 29410 ts_tablet_manager.cc:1939] T e55586542f934e3eabbbed9c6a9d0d30 P de19e6fea21d4611a6e92c99d84cef5c: Deleting consensus metadata
I20260807 22:43:58.377915 29382 tablet_bootstrap.cc:654] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.382062 28819 catalog_manager.cc:4841] TS de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621): tablet e55586542f934e3eabbbed9c6a9d0d30 (table test-table [id=918eeed0b0b04a6daebbc0e9a9922570]) successfully deleted
I20260807 22:43:58.384460 29383 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Bootstrap starting.
I20260807 22:43:58.386518 29383 tablet_bootstrap.cc:654] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.387204 29382 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: No bootstrap required, opened a new log
I20260807 22:43:58.387423 29382 ts_tablet_manager.cc:1397] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.002s
I20260807 22:43:58.389146 29411 ts_tablet_manager.cc:1918] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:58.389473 29411 log.cc:1196] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/e55586542f934e3eabbbed9c6a9d0d30
I20260807 22:43:58.390300 29411 ts_tablet_manager.cc:1939] T e55586542f934e3eabbbed9c6a9d0d30 P 14508720f7164ac59256359e0e134f48: Deleting consensus metadata
I20260807 22:43:58.388578 29382 raft_consensus.cc:348] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.390494 29382 raft_consensus.cc:374] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.390617 29382 raft_consensus.cc:729] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14508720f7164ac59256359e0e134f48, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.391053 29382 consensus_queue.cc:260] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.392800 28822 catalog_manager.cc:4841] TS 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313): tablet e55586542f934e3eabbbed9c6a9d0d30 (table test-table [id=918eeed0b0b04a6daebbc0e9a9922570]) successfully deleted
I20260807 22:43:58.393846 29413 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Bootstrap starting.
I20260807 22:43:58.396006 29413 tablet_bootstrap.cc:654] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.396155 29382 ts_tablet_manager.cc:1428] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260807 22:43:58.397647 29383 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: No bootstrap required, opened a new log
I20260807 22:43:58.397851 29383 ts_tablet_manager.cc:1397] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260807 22:43:58.398897 29383 raft_consensus.cc:348] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.399187 29383 raft_consensus.cc:374] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.399303 29383 raft_consensus.cc:729] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de19e6fea21d4611a6e92c99d84cef5c, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.399684 29383 consensus_queue.cc:260] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.401304 29383 ts_tablet_manager.cc:1428] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:43:58.403051 29413 tablet_bootstrap.cc:492] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: No bootstrap required, opened a new log
I20260807 22:43:58.403331 29413 ts_tablet_manager.cc:1397] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 22:43:58.404374 29413 raft_consensus.cc:348] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.404618 29413 raft_consensus.cc:374] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.404737 29413 raft_consensus.cc:729] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.405074 29413 consensus_queue.cc:260] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.405787 29413 ts_tablet_manager.cc:1428] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 22:43:58.482801 29402 raft_consensus.cc:482] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:58.483067 29402 raft_consensus.cc:504] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.484052 29402 leader_election.cc:288] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:58.484668 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11564109d2d64bd6876739bad9ede908" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39" is_pre_election: true
I20260807 22:43:58.484966 29010 raft_consensus.cc:2454] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:58.484886 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11564109d2d64bd6876739bad9ede908" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c" is_pre_election: true
I20260807 22:43:58.485167 29159 raft_consensus.cc:2454] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:58.485558 29231 leader_election.cc:302] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, 14508720f7164ac59256359e0e134f48; no voters:
I20260807 22:43:58.485899 29402 raft_consensus.cc:2781] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:58.486024 29402 raft_consensus.cc:482] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:58.486181 29402 raft_consensus.cc:3037] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.488978 29402 raft_consensus.cc:504] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.489802 29402 leader_election.cc:288] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Requested vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:58.490293 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11564109d2d64bd6876739bad9ede908" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39"
I20260807 22:43:58.490540 29010 raft_consensus.cc:3037] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.490600 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11564109d2d64bd6876739bad9ede908" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c"
I20260807 22:43:58.490962 29159 raft_consensus.cc:3037] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.493881 29010 raft_consensus.cc:2454] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:58.493922 29159 raft_consensus.cc:2454] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:58.494436 29231 leader_election.cc:302] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, 14508720f7164ac59256359e0e134f48; no voters:
I20260807 22:43:58.494853 29402 raft_consensus.cc:2781] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:58.495031 29402 raft_consensus.cc:686] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Becoming Leader. State: Replica: 14508720f7164ac59256359e0e134f48, State: Running, Role: LEADER
I20260807 22:43:58.495404 29402 consensus_queue.cc:237] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.498582 28836 catalog_manager.cc:5495] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14508720f7164ac59256359e0e134f48 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "14508720f7164ac59256359e0e134f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } health_report { overall_health: HEALTHY } } }
Aug 07 22:43:58 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.190@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:58.549538 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.529664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35027 (local address 127.0.0.1:48400)
0807 22:43:58.530478 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.530509 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.530728 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.531058 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.531062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.531937 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:58.532190 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.532202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.534053 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.534060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.535602 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.535626 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.536902 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.536910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.537117 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.538012 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:58.538038 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:58.541339 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:58.545866 (+ 4527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:58.545887 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:58.545896 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:58.546565 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:58.547005 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:58.547010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:58.547016 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:58.547238 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:58.548158 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:58.548168 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:58.548381 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.548613 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.548983 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":401,"threads_started":1}
I20260807 22:43:58.549696 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.529708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.20.10.190:35027)
0807 22:43:58.530757 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.530762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.530800 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:43:58.530862 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.530865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.530920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.531031 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:58.532404 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.533851 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.535795 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.536489 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.536672 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.541625 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:58.541633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:58.541638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:58.541685 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:58.545662 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:58.546724 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:58.546729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:58.546733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:58.546876 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:58.547442 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:58.547448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:58.547452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:58.547884 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:58.548082 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.548761 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.549229 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":912,"thread_start_us":744,"threads_started":1}
I20260807 22:43:58.556742 28836 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:48400:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:43:58.558106 28836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:58.566695 28984 tablet_service.cc:1467] Processing CreateTablet for tablet 38926b1c0a474f31a23f60f664ef92ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11493a8b1ea343daae8574a1bd4edf4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:58.566931 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 38926b1c0a474f31a23f60f664ef92ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11493a8b1ea343daae8574a1bd4edf4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:58.567050 29281 tablet_service.cc:1467] Processing CreateTablet for tablet 38926b1c0a474f31a23f60f664ef92ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11493a8b1ea343daae8574a1bd4edf4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:58.567420 28984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38926b1c0a474f31a23f60f664ef92ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.567515 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38926b1c0a474f31a23f60f664ef92ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.567659 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38926b1c0a474f31a23f60f664ef92ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:58.576303 29413 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: Bootstrap starting.
I20260807 22:43:58.577008 29382 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: Bootstrap starting.
I20260807 22:43:58.578409 29413 tablet_bootstrap.cc:654] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.579295 29382 tablet_bootstrap.cc:654] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.580297 29383 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: Bootstrap starting.
I20260807 22:43:58.583047 29383 tablet_bootstrap.cc:654] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:58.583552 29413 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: No bootstrap required, opened a new log
I20260807 22:43:58.583817 29413 ts_tablet_manager.cc:1397] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 22:43:58.584843 29413 raft_consensus.cc:348] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.585260 29413 raft_consensus.cc:374] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.585397 29413 raft_consensus.cc:729] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.587159 29413 consensus_queue.cc:260] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.588034 29413 ts_tablet_manager.cc:1428] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260807 22:43:58.587934 29388 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.588548 29388 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.593736 29383 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: No bootstrap required, opened a new log
I20260807 22:43:58.593984 29383 ts_tablet_manager.cc:1397] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260807 22:43:58.594795 29383 raft_consensus.cc:348] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.595125 29383 raft_consensus.cc:374] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.595232 29383 raft_consensus.cc:729] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de19e6fea21d4611a6e92c99d84cef5c, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.595569 29383 consensus_queue.cc:260] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.595980 29382 tablet_bootstrap.cc:492] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: No bootstrap required, opened a new log
I20260807 22:43:58.596220 29382 ts_tablet_manager.cc:1397] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.007s
I20260807 22:43:58.596240 29387 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.596624 29387 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.597157 29382 raft_consensus.cc:348] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.597460 29382 raft_consensus.cc:374] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:58.597432 29388 raft_consensus.cc:482] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:58.597570 29382 raft_consensus.cc:729] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14508720f7164ac59256359e0e134f48, State: Initialized, Role: FOLLOWER
I20260807 22:43:58.597651 29388 raft_consensus.cc:504] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.597853 29382 consensus_queue.cc:260] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.598380 29402 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.598738 29402 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.599651 29388 leader_election.cc:288] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621), 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313)
I20260807 22:43:58.600119 29382 ts_tablet_manager.cc:1428] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260807 22:43:58.601678 29383 ts_tablet_manager.cc:1428] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260807 22:43:58.610828 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.601379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:39645 (local address 127.20.10.130:46621)
0807 22:43:58.601600 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.601607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.601627 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:58.601701 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.601707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.601802 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.601931 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:58.603938 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.605169 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.608175 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.609461 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.609609 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.609714 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.610324 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.610476 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"spinlock_wait_cycles":9472}
I20260807 22:43:58.611723 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.600195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:46621 (local address 127.20.10.129:39645)
0807 22:43:58.600993 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.601025 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.601183 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.603298 (+ 2115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.603305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.603342 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:58.603693 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.603708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.605378 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.605384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.607954 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.607972 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.609925 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.609931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.610109 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.610143 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.611200 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.611365 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":2737,"threads_started":1}
I20260807 22:43:58.612031 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.602455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54341 (local address 127.20.10.131:40313)
0807 22:43:58.602709 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.602716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.604583 (+ 1867us) server_negotiation.cc:407] Connection header received
0807 22:43:58.604925 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.604933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.605068 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.605160 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:58.606217 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.607223 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.609558 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.610640 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.610914 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.610995 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.611373 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.611543 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:43:58.612079 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.600715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:40313 (local address 127.20.10.129:54341)
0807 22:43:58.602407 (+ 1692us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.604495 (+ 2088us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.604701 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.605427 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.605433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.605475 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:58.606036 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.606050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.607434 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.607440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.609364 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.609378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.610921 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.610929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.611157 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.611192 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.611343 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.611604 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"thread_start_us":2379,"threads_started":1}
I20260807 22:43:58.612639 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38926b1c0a474f31a23f60f664ef92ed" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c" is_pre_election: true
I20260807 22:43:58.613052 29159 raft_consensus.cc:2454] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 0.
I20260807 22:43:58.613641 28930 leader_election.cc:302] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:58.613595 29301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38926b1c0a474f31a23f60f664ef92ed" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14508720f7164ac59256359e0e134f48" is_pre_election: true
I20260807 22:43:58.614012 29301 raft_consensus.cc:2454] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 0.
I20260807 22:43:58.614125 29388 raft_consensus.cc:2781] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:58.614235 29388 raft_consensus.cc:482] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:58.614313 29388 raft_consensus.cc:3037] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.617645 29388 raft_consensus.cc:504] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.618960 29388 leader_election.cc:288] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 election: Requested vote from peers de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621), 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313)
I20260807 22:43:58.619124 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38926b1c0a474f31a23f60f664ef92ed" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c"
I20260807 22:43:58.619455 29159 raft_consensus.cc:3037] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.619405 29301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38926b1c0a474f31a23f60f664ef92ed" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14508720f7164ac59256359e0e134f48"
I20260807 22:43:58.619673 29301 raft_consensus.cc:3037] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:58.622998 29159 raft_consensus.cc:2454] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 1.
I20260807 22:43:58.623688 29301 raft_consensus.cc:2454] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 1.
I20260807 22:43:58.623703 28930 leader_election.cc:302] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:58.624351 29388 raft_consensus.cc:2781] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:58.625003 29388 raft_consensus.cc:686] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [term 1 LEADER]: Becoming Leader. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Running, Role: LEADER
I20260807 22:43:58.625532 29388 consensus_queue.cc:237] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:58.627887 29430 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: TxnStatusTablet state changed. Reason: New leader 076d85cfb19943789f6075bb46ee4f39. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.628582 29430 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:58.630113 29432 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:43:58.630399 29432 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:43:58.630251 28836 catalog_manager.cc:5495] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:58.653508 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.645008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.20.10.129:46785)
0807 22:43:58.645253 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.645259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.645364 (+ 105us) server_negotiation.cc:407] Connection header received
0807 22:43:58.645748 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.645756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.645887 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.646060 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:58.646519 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.647890 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.650248 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.651075 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.651227 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.651878 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.652682 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.652849 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.653015 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.653125 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:43:58.653553 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.645014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46785 (local address 127.0.0.1:56180)
0807 22:43:58.645276 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.645300 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.645493 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.646103 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.646109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.646125 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:58.646352 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.646365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.648078 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.648086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.650042 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.650062 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.651280 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.651288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.651519 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.651611 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.652834 (+ 1223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.652862 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.652974 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.653171 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 22:43:58.659323 29159 raft_consensus.cc:1261] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Refusing update from remote peer 076d85cfb19943789f6075bb46ee4f39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.659550 29301 raft_consensus.cc:1261] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Refusing update from remote peer 076d85cfb19943789f6075bb46ee4f39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.660617 29430 consensus_queue.cc:1035] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.661088 29388 consensus_queue.cc:1035] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.675174 29402 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: TxnStatusTablet state changed. Reason: New leader 076d85cfb19943789f6075bb46ee4f39. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.675575 29402 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.677040 29387 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: TxnStatusTablet state changed. Reason: New leader 076d85cfb19943789f6075bb46ee4f39. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.677528 29387 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.680593 29430 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.680917 29430 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:58.681200 29430 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.681479 29430 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 076d85cfb19943789f6075bb46ee4f39: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:58.685207 29402 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.685626 29402 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P 14508720f7164ac59256359e0e134f48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.693121 29387 tablet_replica.cc:439] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } } }
I20260807 22:43:58.693441 29387 tablet_replica.cc:442] T 38926b1c0a474f31a23f60f664ef92ed P de19e6fea21d4611a6e92c99d84cef5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:58.713680 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.704651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:40371 (local address 127.20.10.129:46785)
0807 22:43:58.704952 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.704961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.705816 (+ 855us) server_negotiation.cc:407] Connection header received
0807 22:43:58.706090 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.706097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.706237 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.706350 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:58.707183 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.708492 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.710976 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.711894 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.712049 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.712677 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.712842 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.712962 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.713203 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.713307 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 22:43:58.714908 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.704835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46785 (local address 127.20.10.131:40371)
0807 22:43:58.705653 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.705680 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.705907 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.706403 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.706409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.706424 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:58.706953 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.706970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.708695 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.708704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.710748 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.710769 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.712127 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.712134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.712351 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.712446 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.713030 (+ 584us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.713045 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.714024 (+ 979us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.714209 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":384,"threads_started":1}
I20260807 22:43:58.731791 29159 raft_consensus.cc:1261] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.732211 29010 raft_consensus.cc:1261] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:58.732673 29390 consensus_queue.cc:1035] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.732975 29402 consensus_queue.cc:1035] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:58.805716 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.797710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:43497 (local address 127.20.10.131:40313)
0807 22:43:58.797956 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.797963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.798432 (+ 469us) server_negotiation.cc:407] Connection header received
0807 22:43:58.798724 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.798732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.798836 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.798954 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:58.799924 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.800801 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.803194 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.803832 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.803987 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.804712 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.804883 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.804960 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.805236 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.805343 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 22:43:58.806751 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.797502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:40313 (local address 127.20.10.129:43497)
0807 22:43:58.798303 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:58.798327 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:58.798524 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:58.799203 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:58.799207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:58.799220 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:58.799668 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.799684 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.801175 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.801180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.803025 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:58.803040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.804052 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.804061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:58.804261 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.804419 (+ 158us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:58.805065 (+ 646us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:58.805075 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:43:58.806035 (+ 960us) client_negotiation.cc:240] Negotiation successful
0807 22:43:58.806217 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":414,"threads_started":1}
I20260807 22:43:58.855861 28836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:58.856156 28836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:58.859493 28836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Sending DeleteTablet for 3 replicas of tablet 11564109d2d64bd6876739bad9ede908
I20260807 22:43:58.860632 28983 tablet_service.cc:1514] Processing DeleteTablet for tablet 11564109d2d64bd6876739bad9ede908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56172
I20260807 22:43:58.861061 29409 tablet_replica.cc:330] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: stopping tablet replica
I20260807 22:43:58.861078 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet 11564109d2d64bd6876739bad9ede908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47482
I20260807 22:43:58.861323 29409 raft_consensus.cc:2229] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:58.861392 29410 tablet_replica.cc:330] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: stopping tablet replica
I20260807 22:43:58.861567 29409 raft_consensus.cc:2258] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.861622 29410 raft_consensus.cc:2229] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:58.861886 29410 raft_consensus.cc:2258] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.862479 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet 11564109d2d64bd6876739bad9ede908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:58 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:37428
I20260807 22:43:58.862846 29411 tablet_replica.cc:330] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: stopping tablet replica
I20260807 22:43:58.863065 29411 raft_consensus.cc:2229] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:58.863088 29409 ts_tablet_manager.cc:1905] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.863456 29411 raft_consensus.cc:2258] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:58.864706 29411 ts_tablet_manager.cc:1905] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.871786 29410 ts_tablet_manager.cc:1905] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:58.871878 29409 ts_tablet_manager.cc:1918] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:58.872196 29409 log.cc:1196] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/11564109d2d64bd6876739bad9ede908
I20260807 22:43:58.872829 29409 ts_tablet_manager.cc:1939] T 11564109d2d64bd6876739bad9ede908 P 076d85cfb19943789f6075bb46ee4f39: Deleting consensus metadata
I20260807 22:43:58.874271 28820 catalog_manager.cc:4841] TS 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785): tablet 11564109d2d64bd6876739bad9ede908 (table test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]) successfully deleted
I20260807 22:43:58.878381 29410 ts_tablet_manager.cc:1918] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:58.878595 29410 log.cc:1196] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/11564109d2d64bd6876739bad9ede908
I20260807 22:43:58.879109 29410 ts_tablet_manager.cc:1939] T 11564109d2d64bd6876739bad9ede908 P de19e6fea21d4611a6e92c99d84cef5c: Deleting consensus metadata
I20260807 22:43:58.880405 28819 catalog_manager.cc:4841] TS de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621): tablet 11564109d2d64bd6876739bad9ede908 (table test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]) successfully deleted
I20260807 22:43:58.888569 29411 ts_tablet_manager.cc:1918] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:58.888794 29411 log.cc:1196] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/11564109d2d64bd6876739bad9ede908
I20260807 22:43:58.889248 29411 ts_tablet_manager.cc:1939] T 11564109d2d64bd6876739bad9ede908 P 14508720f7164ac59256359e0e134f48: Deleting consensus metadata
I20260807 22:43:58.890632 28822 catalog_manager.cc:4841] TS 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313): tablet 11564109d2d64bd6876739bad9ede908 (table test-table [id=09bc166d2ecf4b148594d7c964ca3ce2]) successfully deleted
Aug 07 22:43:58 dist-test-slave-csg5 krb5kdc[28788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:43:58 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:58 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:43:58.976157 29423 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:58.960590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.20.10.190:35027)
0807 22:43:58.960873 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.960883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.961744 (+ 861us) server_negotiation.cc:407] Connection header received
0807 22:43:58.961949 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.961955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.962050 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.962166 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:58.964007 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.965235 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.967200 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.967907 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.968055 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.975812 (+ 7757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48416: BlockingRecv error: recv got EOF from 127.0.0.1:48416 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 22:43:58 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM
I20260807 22:43:58.997265 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:58.979837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.20.10.190:35027)
0807 22:43:58.980040 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:58.980047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:58.980503 (+ 456us) server_negotiation.cc:407] Connection header received
0807 22:43:58.980675 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:58.980681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:58.980772 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:58.980892 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:58.982456 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.983624 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.985174 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:58.985817 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:58.985944 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:58.991017 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:58.991026 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:58.991030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:58.991089 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:58.994175 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:58.995029 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:58.995040 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:58.995042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:58.995231 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:58.995684 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:58.995689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:58.995693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:58.995993 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:58.996291 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:58.996619 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 22:43:58.996896 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:43:59.002069 28836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:59.002877 28836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:59.010371 28983 tablet_service.cc:1467] Processing CreateTablet for tablet 836ea7cf37b94701adbd1487ec30555b (DEFAULT_TABLE table=test-table [id=cb63d509257b4235bb9e2130326793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.010485 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 836ea7cf37b94701adbd1487ec30555b (DEFAULT_TABLE table=test-table [id=cb63d509257b4235bb9e2130326793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.010694 29281 tablet_service.cc:1467] Processing CreateTablet for tablet 836ea7cf37b94701adbd1487ec30555b (DEFAULT_TABLE table=test-table [id=cb63d509257b4235bb9e2130326793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.011065 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836ea7cf37b94701adbd1487ec30555b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.011068 28983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836ea7cf37b94701adbd1487ec30555b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.011123 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836ea7cf37b94701adbd1487ec30555b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.018261 29383 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Bootstrap starting.
I20260807 22:43:59.018335 29382 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Bootstrap starting.
I20260807 22:43:59.020205 29383 tablet_bootstrap.cc:654] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.020283 29382 tablet_bootstrap.cc:654] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.023047 29383 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: No bootstrap required, opened a new log
I20260807 22:43:59.023240 29383 ts_tablet_manager.cc:1397] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:59.023940 29383 raft_consensus.cc:348] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.024224 29383 raft_consensus.cc:374] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.024339 29383 raft_consensus.cc:729] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de19e6fea21d4611a6e92c99d84cef5c, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.024410 29413 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Bootstrap starting.
I20260807 22:43:59.025370 29383 consensus_queue.cc:260] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.027036 29413 tablet_bootstrap.cc:654] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.031700 29383 ts_tablet_manager.cc:1428] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260807 22:43:59.032172 29382 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: No bootstrap required, opened a new log
I20260807 22:43:59.032400 29382 ts_tablet_manager.cc:1397] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.007s
I20260807 22:43:59.035076 29382 raft_consensus.cc:348] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.035449 29382 raft_consensus.cc:374] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.035563 29382 raft_consensus.cc:729] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14508720f7164ac59256359e0e134f48, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.035909 29382 consensus_queue.cc:260] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.036639 29382 ts_tablet_manager.cc:1428] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260807 22:43:59.037533 29413 tablet_bootstrap.cc:492] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: No bootstrap required, opened a new log
I20260807 22:43:59.037786 29413 ts_tablet_manager.cc:1397] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260807 22:43:59.039101 29413 raft_consensus.cc:348] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.039584 29413 raft_consensus.cc:374] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.039691 29413 raft_consensus.cc:729] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.040014 29413 consensus_queue.cc:260] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.040704 29413 ts_tablet_manager.cc:1428] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:43:59.110771 29393 raft_consensus.cc:482] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:59.111120 29393 raft_consensus.cc:504] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.112028 29393 leader_election.cc:288] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:59.112871 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836ea7cf37b94701adbd1487ec30555b" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c" is_pre_election: true
I20260807 22:43:59.112900 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836ea7cf37b94701adbd1487ec30555b" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39" is_pre_election: true
I20260807 22:43:59.113195 29158 raft_consensus.cc:2454] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:59.113222 29010 raft_consensus.cc:2454] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 0.
I20260807 22:43:59.113859 29231 leader_election.cc:302] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, 14508720f7164ac59256359e0e134f48, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:59.114307 29393 raft_consensus.cc:2781] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:59.114454 29393 raft_consensus.cc:482] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:59.114586 29393 raft_consensus.cc:3037] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.118261 29393 raft_consensus.cc:504] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.119845 29010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836ea7cf37b94701adbd1487ec30555b" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076d85cfb19943789f6075bb46ee4f39"
I20260807 22:43:59.120154 29010 raft_consensus.cc:3037] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.120391 29393 leader_election.cc:288] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Requested vote from peers 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785), de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621)
I20260807 22:43:59.121069 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836ea7cf37b94701adbd1487ec30555b" candidate_uuid: "14508720f7164ac59256359e0e134f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c"
I20260807 22:43:59.121330 29158 raft_consensus.cc:3037] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.123582 29158 raft_consensus.cc:2454] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:59.124176 29230 leader_election.cc:302] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14508720f7164ac59256359e0e134f48, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:59.124163 29010 raft_consensus.cc:2454] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14508720f7164ac59256359e0e134f48 in term 1.
I20260807 22:43:59.124533 29393 raft_consensus.cc:2781] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:59.124758 29393 raft_consensus.cc:686] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Becoming Leader. State: Replica: 14508720f7164ac59256359e0e134f48, State: Running, Role: LEADER
I20260807 22:43:59.125036 29393 consensus_queue.cc:237] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.128377 28835 catalog_manager.cc:5495] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14508720f7164ac59256359e0e134f48 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "14508720f7164ac59256359e0e134f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } health_report { overall_health: HEALTHY } } }
I20260807 22:43:59.151528 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.145422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.20.10.131:40313)
0807 22:43:59.146282 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.146290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.146316 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:59.146397 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.146403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.146506 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.146751 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:59.147152 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.148014 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.149479 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.149961 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.150109 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.150630 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:59.150833 (+ 203us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:59.150942 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.151112 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.151205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631}
I20260807 22:43:59.154928 29158 raft_consensus.cc:1261] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:59.154953 29010 raft_consensus.cc:1261] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Refusing update from remote peer 14508720f7164ac59256359e0e134f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:59.155676 29402 consensus_queue.cc:1035] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:59.156041 29393 consensus_queue.cc:1035] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:59.171509 28836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:59.171818 28836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:59.176570 28836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Sending DeleteTablet for 3 replicas of tablet 836ea7cf37b94701adbd1487ec30555b
I20260807 22:43:59.177714 28983 tablet_service.cc:1514] Processing DeleteTablet for tablet 836ea7cf37b94701adbd1487ec30555b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56172
I20260807 22:43:59.177731 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet 836ea7cf37b94701adbd1487ec30555b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:37428
I20260807 22:43:59.178069 29411 tablet_replica.cc:330] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: stopping tablet replica
I20260807 22:43:59.178247 29409 tablet_replica.cc:330] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: stopping tablet replica
I20260807 22:43:59.178297 29411 raft_consensus.cc:2229] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:59.178463 29409 raft_consensus.cc:2229] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:59.178766 29411 raft_consensus.cc:2258] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.178784 29409 raft_consensus.cc:2258] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.179806 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet 836ea7cf37b94701adbd1487ec30555b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47482
I20260807 22:43:59.180042 29131 tablet_service.cc:1514] Processing DeleteTablet for tablet 836ea7cf37b94701adbd1487ec30555b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47482
I20260807 22:43:59.180269 29410 tablet_replica.cc:330] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: stopping tablet replica
I20260807 22:43:59.180467 29410 raft_consensus.cc:2229] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:59.180724 29410 raft_consensus.cc:2258] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.181882 29410 ts_tablet_manager.cc:1905] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:59.181710 28833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:43:59.181979 29411 ts_tablet_manager.cc:1905] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 22:43:59.182468 28833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:59.187980 29409 ts_tablet_manager.cc:1905] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:59.190085 29410 ts_tablet_manager.cc:1918] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:59.190250 29410 log.cc:1196] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/836ea7cf37b94701adbd1487ec30555b
I20260807 22:43:59.190985 29410 ts_tablet_manager.cc:1939] T 836ea7cf37b94701adbd1487ec30555b P de19e6fea21d4611a6e92c99d84cef5c: Deleting consensus metadata
I20260807 22:43:59.193602 28819 catalog_manager.cc:4841] TS de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621): tablet 836ea7cf37b94701adbd1487ec30555b (table test-table [id=cb63d509257b4235bb9e2130326793ad]) successfully deleted
I20260807 22:43:59.195241 29131 tablet_service.cc:1467] Processing CreateTablet for tablet f83ef0b3fd644e538723bb40e9c87695 (DEFAULT_TABLE table=test-table [id=411f9c3eaca049568422e3ffd650b737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.195772 29131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ef0b3fd644e538723bb40e9c87695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.197275 29411 ts_tablet_manager.cc:1918] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:59.197443 29411 log.cc:1196] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/836ea7cf37b94701adbd1487ec30555b
I20260807 22:43:59.197906 29411 ts_tablet_manager.cc:1939] T 836ea7cf37b94701adbd1487ec30555b P 14508720f7164ac59256359e0e134f48: Deleting consensus metadata
I20260807 22:43:59.199486 29281 tablet_service.cc:1467] Processing CreateTablet for tablet f83ef0b3fd644e538723bb40e9c87695 (DEFAULT_TABLE table=test-table [id=411f9c3eaca049568422e3ffd650b737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.199801 28983 tablet_service.cc:1467] Processing CreateTablet for tablet f83ef0b3fd644e538723bb40e9c87695 (DEFAULT_TABLE table=test-table [id=411f9c3eaca049568422e3ffd650b737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:59.200042 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ef0b3fd644e538723bb40e9c87695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.200155 29409 ts_tablet_manager.cc:1918] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:59.200250 28983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ef0b3fd644e538723bb40e9c87695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:59.200474 28822 catalog_manager.cc:4841] TS 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313): tablet 836ea7cf37b94701adbd1487ec30555b (table test-table [id=cb63d509257b4235bb9e2130326793ad]) successfully deleted
I20260807 22:43:59.202876 29409 log.cc:1196] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/836ea7cf37b94701adbd1487ec30555b
W20260807 22:43:59.203418 28819 catalog_manager.cc:4808] TS de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621): delete failed for tablet 836ea7cf37b94701adbd1487ec30555b because the tablet was not found. No further retry: Not found: Tablet not found: 836ea7cf37b94701adbd1487ec30555b
I20260807 22:43:59.205125 29409 ts_tablet_manager.cc:1939] T 836ea7cf37b94701adbd1487ec30555b P 076d85cfb19943789f6075bb46ee4f39: Deleting consensus metadata
I20260807 22:43:59.206434 28820 catalog_manager.cc:4841] TS 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785): tablet 836ea7cf37b94701adbd1487ec30555b (table test-table [id=cb63d509257b4235bb9e2130326793ad]) successfully deleted
I20260807 22:43:59.207918 29413 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Bootstrap starting.
I20260807 22:43:59.209251 29383 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Bootstrap starting.
I20260807 22:43:59.209934 29413 tablet_bootstrap.cc:654] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.211503 29383 tablet_bootstrap.cc:654] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.211853 29382 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Bootstrap starting.
I20260807 22:43:59.214228 29382 tablet_bootstrap.cc:654] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:59.219748 29382 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: No bootstrap required, opened a new log
I20260807 22:43:59.219973 29382 ts_tablet_manager.cc:1397] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260807 22:43:59.221062 29382 raft_consensus.cc:348] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.221407 29382 raft_consensus.cc:374] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.221503 29382 raft_consensus.cc:729] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14508720f7164ac59256359e0e134f48, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.221799 29382 consensus_queue.cc:260] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.222815 29413 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: No bootstrap required, opened a new log
I20260807 22:43:59.223033 29413 ts_tablet_manager.cc:1397] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.002s
I20260807 22:43:59.223348 29382 ts_tablet_manager.cc:1428] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 22:43:59.224224 29413 raft_consensus.cc:348] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.224507 29413 raft_consensus.cc:374] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.224570 29413 raft_consensus.cc:729] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.224910 29413 consensus_queue.cc:260] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.225930 29383 tablet_bootstrap.cc:492] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: No bootstrap required, opened a new log
I20260807 22:43:59.226212 29383 ts_tablet_manager.cc:1397] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.003s
I20260807 22:43:59.226586 29413 ts_tablet_manager.cc:1428] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:59.228065 29383 raft_consensus.cc:348] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.228401 29383 raft_consensus.cc:374] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:59.228508 29383 raft_consensus.cc:729] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de19e6fea21d4611a6e92c99d84cef5c, State: Initialized, Role: FOLLOWER
I20260807 22:43:59.228871 29383 consensus_queue.cc:260] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.229533 29383 ts_tablet_manager.cc:1428] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 22:43:59.362124 29440 raft_consensus.cc:482] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:59.362377 29440 raft_consensus.cc:504] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.363265 29440 leader_election.cc:288] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621), 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313)
I20260807 22:43:59.364077 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ef0b3fd644e538723bb40e9c87695" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c" is_pre_election: true
I20260807 22:43:59.364076 29300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ef0b3fd644e538723bb40e9c87695" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14508720f7164ac59256359e0e134f48" is_pre_election: true
I20260807 22:43:59.364392 29300 raft_consensus.cc:2454] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 0.
I20260807 22:43:59.364391 29158 raft_consensus.cc:2454] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 0.
I20260807 22:43:59.364976 28930 leader_election.cc:302] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, 14508720f7164ac59256359e0e134f48, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:59.365482 29440 raft_consensus.cc:2781] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:59.365638 29440 raft_consensus.cc:482] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:59.365744 29440 raft_consensus.cc:3037] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.367976 29440 raft_consensus.cc:504] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.368718 29440 leader_election.cc:288] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 election: Requested vote from peers de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621), 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313)
I20260807 22:43:59.369438 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ef0b3fd644e538723bb40e9c87695" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de19e6fea21d4611a6e92c99d84cef5c"
I20260807 22:43:59.369448 29300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ef0b3fd644e538723bb40e9c87695" candidate_uuid: "076d85cfb19943789f6075bb46ee4f39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14508720f7164ac59256359e0e134f48"
I20260807 22:43:59.369686 29158 raft_consensus.cc:3037] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.369678 29300 raft_consensus.cc:3037] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:59.372165 29158 raft_consensus.cc:2454] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 1.
I20260807 22:43:59.372296 29300 raft_consensus.cc:2454] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d85cfb19943789f6075bb46ee4f39 in term 1.
I20260807 22:43:59.372792 28930 leader_election.cc:302] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076d85cfb19943789f6075bb46ee4f39, de19e6fea21d4611a6e92c99d84cef5c; no voters:
I20260807 22:43:59.373191 29440 raft_consensus.cc:2781] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:59.373643 29440 raft_consensus.cc:686] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 1 LEADER]: Becoming Leader. State: Replica: 076d85cfb19943789f6075bb46ee4f39, State: Running, Role: LEADER
I20260807 22:43:59.374004 29440 consensus_queue.cc:237] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } }
I20260807 22:43:59.376988 28833 catalog_manager.cc:5495] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "076d85cfb19943789f6075bb46ee4f39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076d85cfb19943789f6075bb46ee4f39" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 46785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:59.433455 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.426345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56192 (local address 127.20.10.129:46785)
0807 22:43:59.426986 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.426991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.427010 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:43:59.427077 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.427081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.427183 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.427437 (+ 254us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:59.427963 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.429114 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.430948 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.431396 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.431509 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.432426 (+ 917us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:59.432551 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:59.432634 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.432958 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.433043 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":312,"threads_started":1}
I20260807 22:43:59.450809 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.444392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:42399 (local address 127.20.10.129:46785)
0807 22:43:59.444559 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.444564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.445153 (+ 589us) server_negotiation.cc:407] Connection header received
0807 22:43:59.445272 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.445277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.445370 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.445510 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:59.446035 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.447144 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.448772 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.449275 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.449391 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.449935 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:59.450042 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:59.450113 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.450434 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.450505 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:43:59.450961 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.444083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:46785 (local address 127.20.10.129:42399)
0807 22:43:59.444996 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:59.445012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:59.445153 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:59.445528 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:59.445533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:59.445551 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:59.445855 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:59.445865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.447352 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.447355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:59.448482 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:59.448495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.449454 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.449459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:59.449738 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.449785 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:59.450187 (+ 402us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:59.450203 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 22:43:59.450313 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 22:43:59.450452 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":679,"threads_started":1}
I20260807 22:43:59.460238 29300 raft_consensus.cc:1261] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Refusing update from remote peer 076d85cfb19943789f6075bb46ee4f39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:59.460304 29158 raft_consensus.cc:1261] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Refusing update from remote peer 076d85cfb19943789f6075bb46ee4f39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:59.460927 29480 consensus_queue.cc:1035] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de19e6fea21d4611a6e92c99d84cef5c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:59.461282 29475 consensus_queue.cc:1035] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14508720f7164ac59256359e0e134f48" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 40313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:59.540875 28833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:59.541222 28833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:59.545487 28833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c7a79c953e946eb8b458c3ccd730856: Sending DeleteTablet for 3 replicas of tablet f83ef0b3fd644e538723bb40e9c87695
I20260807 22:43:59.546602 29131 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ef0b3fd644e538723bb40e9c87695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47482
I20260807 22:43:59.546967 29410 tablet_replica.cc:330] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: stopping tablet replica
I20260807 22:43:59.547195 29410 raft_consensus.cc:2229] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:59.547439 29410 raft_consensus.cc:2258] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.547809 28984 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ef0b3fd644e538723bb40e9c87695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56172
I20260807 22:43:59.548182 29409 tablet_replica.cc:330] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: stopping tablet replica
I20260807 22:43:59.548466 29409 raft_consensus.cc:2229] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:59.549060 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ef0b3fd644e538723bb40e9c87695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:59 UTC) from {username='oryx', principal='oryx/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:37428
I20260807 22:43:59.549191 29409 raft_consensus.cc:2258] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.549542 29411 tablet_replica.cc:330] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: stopping tablet replica
I20260807 22:43:59.549788 29411 raft_consensus.cc:2229] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:59.549999 29411 raft_consensus.cc:2258] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:59.550277 29410 ts_tablet_manager.cc:1905] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:59.551911 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 28913
I20260807 22:43:59.552250 29409 ts_tablet_manager.cc:1905] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:59.559474 29410 ts_tablet_manager.cc:1918] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:59.560374 29410 log.cc:1196] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/f83ef0b3fd644e538723bb40e9c87695
I20260807 22:43:59.561244 29410 ts_tablet_manager.cc:1939] T f83ef0b3fd644e538723bb40e9c87695 P de19e6fea21d4611a6e92c99d84cef5c: Deleting consensus metadata
I20260807 22:43:59.562888 28819 catalog_manager.cc:4841] TS de19e6fea21d4611a6e92c99d84cef5c (127.20.10.130:46621): tablet f83ef0b3fd644e538723bb40e9c87695 (table test-table [id=411f9c3eaca049568422e3ffd650b737]) successfully deleted
I20260807 22:43:59.563762 29411 ts_tablet_manager.cc:1905] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:59.568202 29409 ts_tablet_manager.cc:1918] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:59.568549 29409 log.cc:1196] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/f83ef0b3fd644e538723bb40e9c87695
I20260807 22:43:59.569227 29409 ts_tablet_manager.cc:1939] T f83ef0b3fd644e538723bb40e9c87695 P 076d85cfb19943789f6075bb46ee4f39: Deleting consensus metadata
I20260807 22:43:59.571053 28820 catalog_manager.cc:4841] TS 076d85cfb19943789f6075bb46ee4f39 (127.20.10.129:46785): tablet f83ef0b3fd644e538723bb40e9c87695 (table test-table [id=411f9c3eaca049568422e3ffd650b737]) successfully deleted
I20260807 22:43:59.572759 29411 ts_tablet_manager.cc:1918] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:59.573051 29411 log.cc:1196] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/f83ef0b3fd644e538723bb40e9c87695
I20260807 22:43:59.573945 29411 ts_tablet_manager.cc:1939] T f83ef0b3fd644e538723bb40e9c87695 P 14508720f7164ac59256359e0e134f48: Deleting consensus metadata
I20260807 22:43:59.575752 28822 catalog_manager.cc:4841] TS 14508720f7164ac59256359e0e134f48 (127.20.10.131:40313): tablet f83ef0b3fd644e538723bb40e9c87695 (table test-table [id=411f9c3eaca049568422e3ffd650b737]) successfully deleted
Aug 07 22:43:59 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.129@KRBTEST.COM
I20260807 22:43:59.596323 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.552819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.20.10.129:46785)
0807 22:43:59.553033 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.553040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.571692 (+ 18652us) server_negotiation.cc:407] Connection header received
0807 22:43:59.572194 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.572203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.572463 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.572563 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:59.575924 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.577499 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.579323 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.579966 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.580145 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.586013 (+ 5868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:59.586034 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:59.586052 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:59.586098 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:59.591608 (+ 5510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.592702 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.592710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.592713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.593240 (+ 527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.593889 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.593899 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.593906 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.594517 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:59.594980 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.595305 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.595718 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:43:59.597596 29056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:59.750849 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 28913
I20260807 22:43:59.784454 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29065
Aug 07 22:43:59 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.130@KRBTEST.COM
I20260807 22:43:59.807742 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.785602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.20.10.130:46621)
0807 22:43:59.786397 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.786405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.786431 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:59.786513 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.786518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.786632 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.786808 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:59.789172 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.790543 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.793702 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.794324 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.794484 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.800654 (+ 6170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:59.800674 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:59.800691 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:59.800736 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:59.804200 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.805047 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.805055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.805057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.805221 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.805696 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.805701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.805703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.806273 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:59.806521 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.807078 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.807271 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":407,"threads_started":1}
I20260807 22:43:59.808534 29203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:43:59.916215 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29065
I20260807 22:43:59.943298 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29213
Aug 07 22:43:59 dist-test-slave-csg5 krb5kdc[28788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.10.131@KRBTEST.COM
I20260807 22:43:59.962872 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:59.944199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.20.10.131:40313)
0807 22:43:59.944800 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:59.944804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:59.944822 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:43:59.944889 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:59.944893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:59.944957 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:59.945058 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:59.946996 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.947804 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.949281 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:59.949844 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:59.949972 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:59.955101 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:59.955241 (+ 140us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:59.955259 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:59.955318 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:59.958957 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.959968 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.959976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.959979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.960148 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:59.960771 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:59.960775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:59.960777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:59.961196 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:59.961590 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:59.962239 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 22:43:59.962489 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":368,"threads_started":1}
I20260807 22:43:59.963824 29349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:00.103752 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29213
I20260807 22:44:00.133978 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 28802
I20260807 22:44:00.135666 28879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:00.264730 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 28802
2026-08-07T22:44:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7864 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:44:00 dist-test-slave-csg5 krb5kdc[29496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:44:00 dist-test-slave-csg5 krb5kdc[29496](info): set up 2 sockets
Aug 07 22:44:00 dist-test-slave-csg5 krb5kdc[29496](info): commencing operation
krb5kdc: starting...
W20260807 22:44:01.587920 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:44:01 dist-test-slave-csg5 krb5kdc[29496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:44:01.923995 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35211
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:02.331449 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:02.331796 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:02.331959 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:02.340510 29509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:02.340667 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:02.340813 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:02.340956 29509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:02.341029 29509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:02.355445 29509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:02.356887 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:02.358453 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:02.373162 29517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:02.373466 29514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:02.373435 29515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:02.373713 29509 server_base.cc:1034] running on GCE node
I20260807 22:44:02.375201 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:02.377106 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:02.378501 29509 hybrid_clock.cc:648] HybridClock initialized: now 1786142642378436 us; error 57 us; skew 500 ppm
I20260807 22:44:02.379048 29509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T22:44:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2187 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:44:02 dist-test-slave-csg5 krb5kdc[29523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:44:02 dist-test-slave-csg5 krb5kdc[29523](info): set up 2 sockets
Aug 07 22:44:02 dist-test-slave-csg5 krb5kdc[29523](info): commencing operation
krb5kdc: starting...
W20260807 22:44:03.753919 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:44:03 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:44:04.087692 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42065
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:04.471535 29538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:04.471976 29538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:04.472204 29538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:04.481201 29538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:04.481374 29538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:04.481498 29538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:04.481588 29538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:04.481668 29538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:04.493675 29538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:42065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:04.495440 29538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:04.496999 29538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:04.511200 29544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:04.511323 29546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:04.511364 29543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:04.512807 29538 server_base.cc:1034] running on GCE node
I20260807 22:44:04.513934 29538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:04.515641 29538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:04.516896 29538 hybrid_clock.cc:648] HybridClock initialized: now 1786142644516820 us; error 63 us; skew 500 ppm
I20260807 22:44:04.517385 29538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:04 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142644, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:04.521387 29538 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:44:04.523262 29538 webserver.cc:460] Webserver started at http://127.20.10.190:37311/ using document root <none> and password file <none>
I20260807 22:44:04.524041 29538 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:04.524158 29538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:04.524464 29538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:04.528213 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "36b31fbe3dc54e6cb416fe36f53f7c40"
format_stamp: "Formatted at 2026-08-07 22:44:04 on dist-test-slave-csg5"
server_key: "e8a3c378e4a3e47883e55b248d389c1b"
server_key_iv: "03c8225db1a41a41fe0236e1df9e6df1"
server_key_version: "encryptionkey@0"
I20260807 22:44:04.529183 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "36b31fbe3dc54e6cb416fe36f53f7c40"
format_stamp: "Formatted at 2026-08-07 22:44:04 on dist-test-slave-csg5"
server_key: "e8a3c378e4a3e47883e55b248d389c1b"
server_key_iv: "03c8225db1a41a41fe0236e1df9e6df1"
server_key_version: "encryptionkey@0"
I20260807 22:44:04.536317 29538 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 22:44:04.541033 29555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:04.542888 29538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:44:04.543134 29538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "36b31fbe3dc54e6cb416fe36f53f7c40"
format_stamp: "Formatted at 2026-08-07 22:44:04 on dist-test-slave-csg5"
server_key: "e8a3c378e4a3e47883e55b248d389c1b"
server_key_iv: "03c8225db1a41a41fe0236e1df9e6df1"
server_key_version: "encryptionkey@0"
I20260807 22:44:04.543452 29538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:04.598192 29538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:04.604239 29538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:04.604612 29538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:04.628799 29538 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:42065
I20260807 22:44:04.628831 29608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:42065 every 8 connection(s)
I20260807 22:44:04.630489 29538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:04.631264 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29538
I20260807 22:44:04.631597 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:44:04.632436 20522 external_mini_cluster.cc:1428] Setting key c289e952ce89ce52a9cf710ea712b631
I20260807 22:44:04.636024 29609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:44:04 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:04.649886 29609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Bootstrap starting.
I20260807 22:44:04.654279 29609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:04.655817 29609 log.cc:824] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:04.660032 29609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: No bootstrap required, opened a new log
I20260807 22:44:04.661536 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:04.634895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.20.10.190:42065)
0807 22:44:04.635791 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:04.635809 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:04.635861 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:44:04.638377 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:04.638408 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:04.638953 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:04.639445 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:04.641136 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:04.643286 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:04.644804 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:04.645932 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:04.646141 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:04.651214 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:04.651226 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:04.651247 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:04.651309 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:04.655791 (+ 4482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:04.656883 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:04.656891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:04.656901 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:04.657091 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:04.657752 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:04.657758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:04.657761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:04.658559 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:04.658931 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:04.659508 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 22:44:04.660000 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":358,"threads_started":1}
I20260807 22:44:04.670252 29609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:04.670764 29609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:04.670881 29609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b31fbe3dc54e6cb416fe36f53f7c40, State: Initialized, Role: FOLLOWER
I20260807 22:44:04.671494 29609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:04.671725 29609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:04.671836 29609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:44:04.671993 29609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:04.674569 29609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:04.675374 29609 leader_election.cc:302] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36b31fbe3dc54e6cb416fe36f53f7c40; no voters:
I20260807 22:44:04.676122 29609 leader_election.cc:288] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:44:04.676317 29614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:04.677240 29614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 LEADER]: Becoming Leader. State: Replica: 36b31fbe3dc54e6cb416fe36f53f7c40, State: Running, Role: LEADER
I20260807 22:44:04.678000 29614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:04.678740 29609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:04.681643 29617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b31fbe3dc54e6cb416fe36f53f7c40. Latest consensus state: current_term: 1 leader_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } } }
I20260807 22:44:04.682729 29616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } } }
I20260807 22:44:04.683390 29616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:04.685480 29617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:04.686641 29625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:04.693655 29625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:04.706009 29625 catalog_manager.cc:1347] Generated new cluster ID: 47119d9dad7b4a31a122cea7a3496244
I20260807 22:44:04.706185 29625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:04.719321 29625 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:44:04.721432 29625 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:04.734973 29625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Generated new TSK 0
I20260807 22:44:04.736222 29625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:44:04.816345 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:05.212479 29639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:05.212801 29639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:05.212980 29639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:05.220834 29639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:05.220976 29639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:05.221231 29639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:44:05.231729 29639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:05.233300 29639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:05.234871 29639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:05.251195 29644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:05.251132 29645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:05.251160 29647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:05.252461 29639 server_base.cc:1034] running on GCE node
I20260807 22:44:05.253371 29639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:05.254573 29639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:05.255893 29639 hybrid_clock.cc:648] HybridClock initialized: now 1786142645255840 us; error 44 us; skew 500 ppm
I20260807 22:44:05.256292 29639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:05 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142645, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:05.260588 29639 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:44:05.262456 29639 webserver.cc:460] Webserver started at http://127.20.10.129:39031/ using document root <none> and password file <none>
I20260807 22:44:05.263315 29639 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:05.263466 29639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:05.263835 29639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:05.267594 29639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "e38e595d75914e6ea12b4b1d2e01d726"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "b10cabf17462ed31114b36292936958b"
server_key_iv: "bba400d817d67b146ed3b6c232330c6a"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.268730 29639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "e38e595d75914e6ea12b4b1d2e01d726"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "b10cabf17462ed31114b36292936958b"
server_key_iv: "bba400d817d67b146ed3b6c232330c6a"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.275552 29639 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:44:05.280128 29654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.282071 29639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:44:05.282323 29639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "e38e595d75914e6ea12b4b1d2e01d726"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "b10cabf17462ed31114b36292936958b"
server_key_iv: "bba400d817d67b146ed3b6c232330c6a"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.282629 29639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:05.300685 29639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:05.305990 29639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:05.306356 29639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:05.307535 29639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:05.309062 29639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:05.309145 29639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.309342 29639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:05.309415 29639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.350559 29639 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:39409
I20260807 22:44:05.350618 29776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:39409 every 8 connection(s)
I20260807 22:44:05.352043 29639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:05.359591 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29639
I20260807 22:44:05.359930 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:44:05.360584 20522 external_mini_cluster.cc:1428] Setting key 9b2681db5e48c71b3b611c03031cbfa1
Aug 07 22:44:05 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142645, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:05.376717 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:05.355088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:41203 (local address 127.20.10.190:42065)
0807 22:44:05.355603 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:05.355608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:05.356894 (+ 1286us) server_negotiation.cc:407] Connection header received
0807 22:44:05.359455 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:05.359462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:05.359595 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:05.359749 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:05.361853 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.362812 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.364738 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.365213 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.365387 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:05.370139 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:05.370145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:05.370148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:05.370188 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:05.373146 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:05.374054 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:05.374061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:05.374071 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:05.374212 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:05.374748 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:05.374758 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:05.374760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:05.375058 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:05.375241 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:05.376042 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 22:44:05.376326 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":255,"threads_started":1}
I20260807 22:44:05.377811 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:05.355711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:41203)
0807 22:44:05.356668 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:05.356735 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:05.359088 (+ 2353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:05.360154 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:05.360168 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:05.360901 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:05.361636 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:05.361661 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.363024 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.363029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:05.364537 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:05.364551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.365529 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.365539 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:05.365846 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:05.366742 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:05.366771 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:05.369842 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:05.373346 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:05.373357 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:05.373385 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:05.373912 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:05.374340 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:05.374345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:05.374349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:05.374578 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:05.375212 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:05.375222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:05.375633 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 22:44:05.375924 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 22:44:05.376539 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":395,"threads_started":1}
I20260807 22:44:05.379703 29777 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42065
I20260807 22:44:05.380089 29777 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:05.380918 29777 heartbeater.cc:511] Master 127.20.10.190:42065 requested a full tablet report, sending...
I20260807 22:44:05.383574 29572 ts_manager.cc:194] Registered new tserver with Master: e38e595d75914e6ea12b4b1d2e01d726 (127.20.10.129:39409)
I20260807 22:44:05.385941 29572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:41203
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:44:05.431581 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:05.810874 29784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:05.811453 29784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:05.811653 29784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:05.819845 29784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:05.820003 29784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:05.820204 29784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:44:05.830847 29784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:05.832365 29784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:05.833876 29784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:05.848915 29789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:05.848970 29793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:05.849049 29790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:05.850576 29784 server_base.cc:1034] running on GCE node
I20260807 22:44:05.851372 29784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:05.852653 29784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:05.854000 29784 hybrid_clock.cc:648] HybridClock initialized: now 1786142645853896 us; error 72 us; skew 500 ppm
I20260807 22:44:05.854442 29784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:05 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142645, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:05.859736 29784 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:44:05.861764 29784 webserver.cc:460] Webserver started at http://127.20.10.130:34679/ using document root <none> and password file <none>
I20260807 22:44:05.862607 29784 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:05.862854 29784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:05.863246 29784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:05.866972 29784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "21831f56c35846dda37ce616c39aee38"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "d525b9de842a82fed128cd4a0139499d"
server_key_iv: "7c3b17fc103841603897a3be0bcdb6f8"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.868000 29784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "21831f56c35846dda37ce616c39aee38"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "d525b9de842a82fed128cd4a0139499d"
server_key_iv: "7c3b17fc103841603897a3be0bcdb6f8"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.874562 29784 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:44:05.879514 29800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.881305 29784 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:44:05.881528 29784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "21831f56c35846dda37ce616c39aee38"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "d525b9de842a82fed128cd4a0139499d"
server_key_iv: "7c3b17fc103841603897a3be0bcdb6f8"
server_key_version: "encryptionkey@0"
I20260807 22:44:05.881783 29784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:05.901053 29784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:05.906982 29784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:05.907393 29784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:05.908648 29784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:05.910388 29784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:05.910516 29784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.910750 29784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:05.910840 29784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:05.952028 29784 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:34229
I20260807 22:44:05.952085 29920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:34229 every 8 connection(s)
I20260807 22:44:05.953465 29784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:05.955111 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29784
I20260807 22:44:05.955345 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:44:05.955895 20522 external_mini_cluster.cc:1428] Setting key ff0f93f4ae00a8d4fb02e7602b1363b7
Aug 07 22:44:05 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142645, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:05.978542 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:05.956449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49281 (local address 127.20.10.190:42065)
0807 22:44:05.958806 (+ 2357us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:05.958813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:05.958839 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:44:05.960600 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:05.960610 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:05.960713 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:05.960861 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:05.963020 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.964257 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.966280 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.967110 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.967276 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:05.971853 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:05.971859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:05.971862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:05.971904 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:05.974831 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:05.975769 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:05.975782 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:05.975785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:05.975933 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:05.976447 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:05.976451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:05.976453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:05.976751 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:05.976926 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:05.977783 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 22:44:05.978068 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":276,"threads_started":1}
I20260807 22:44:05.980021 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:05.957153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:49281)
0807 22:44:05.958112 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:05.958183 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:05.960269 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:05.961334 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:05.961344 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:05.961987 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:05.962784 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:05.962810 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.964508 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.964513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:05.966060 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:05.966077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:05.967506 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:05.967513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:05.967802 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:05.968746 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:05.968774 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:05.971531 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:05.975031 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:05.975043 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:05.975072 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:05.975604 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:05.976066 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:05.976072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:05.976076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:05.976304 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:05.976903 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:05.976913 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:05.977369 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 22:44:05.977677 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 22:44:05.978315 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":346,"threads_started":1}
I20260807 22:44:05.982087 29921 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42065
I20260807 22:44:05.982566 29921 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:05.983532 29921 heartbeater.cc:511] Master 127.20.10.190:42065 requested a full tablet report, sending...
I20260807 22:44:05.985512 29572 ts_manager.cc:194] Registered new tserver with Master: 21831f56c35846dda37ce616c39aee38 (127.20.10.130:34229)
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:44:05.987008 29572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:49281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:44:06.031337 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:44:06.390389 29777 heartbeater.cc:503] Master 127.20.10.190:42065 was elected leader, sending a full tablet report...
W20260807 22:44:06.421667 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:06.422000 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:06.422150 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:06.430022 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:06.430171 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:06.430390 29928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:44:06.441006 29928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:06.442885 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:06.444694 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:06.462175 29935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:06.462088 29934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:06.462239 29937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:06.463328 29928 server_base.cc:1034] running on GCE node
I20260807 22:44:06.464330 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:06.465597 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:06.466879 29928 hybrid_clock.cc:648] HybridClock initialized: now 1786142646466817 us; error 54 us; skew 500 ppm
I20260807 22:44:06.467290 29928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:06 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142646, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:06.471988 29928 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:44:06.473863 29928 webserver.cc:460] Webserver started at http://127.20.10.131:35147/ using document root <none> and password file <none>
I20260807 22:44:06.474735 29928 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:06.474879 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:06.475204 29928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:06.479347 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "6c73c4a58bc64cc2a061195f83d0915f"
format_stamp: "Formatted at 2026-08-07 22:44:06 on dist-test-slave-csg5"
server_key: "41bdbc08fdfac5f5b33cd27fb68db03f"
server_key_iv: "4db84b569d9acb5d0caa361db871ea95"
server_key_version: "encryptionkey@0"
I20260807 22:44:06.480371 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "6c73c4a58bc64cc2a061195f83d0915f"
format_stamp: "Formatted at 2026-08-07 22:44:06 on dist-test-slave-csg5"
server_key: "41bdbc08fdfac5f5b33cd27fb68db03f"
server_key_iv: "4db84b569d9acb5d0caa361db871ea95"
server_key_version: "encryptionkey@0"
I20260807 22:44:06.487258 29928 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260807 22:44:06.491660 29945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:06.493533 29928 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:44:06.493793 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "6c73c4a58bc64cc2a061195f83d0915f"
format_stamp: "Formatted at 2026-08-07 22:44:06 on dist-test-slave-csg5"
server_key: "41bdbc08fdfac5f5b33cd27fb68db03f"
server_key_iv: "4db84b569d9acb5d0caa361db871ea95"
server_key_version: "encryptionkey@0"
I20260807 22:44:06.494149 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:06.522377 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:06.527946 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:06.528259 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:06.529385 29928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:06.531076 29928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:06.531163 29928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:06.531265 29928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:06.531306 29928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:06.577039 29928 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:36131
I20260807 22:44:06.577087 30070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:36131 every 8 connection(s)
I20260807 22:44:06.578526 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:06.579248 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 29928
I20260807 22:44:06.579509 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:44:06.580277 20522 external_mini_cluster.cc:1428] Setting key 6b979622d7d0efdf9916f8559ca79a15
Aug 07 22:44:06 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142646, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:06.606424 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:06.582007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:43021 (local address 127.20.10.190:42065)
0807 22:44:06.582908 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:06.582921 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:06.584800 (+ 1879us) server_negotiation.cc:407] Connection header received
0807 22:44:06.587075 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:06.587081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:06.587181 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:06.587347 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:06.589633 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.590490 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.592882 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.593387 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.593567 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:06.598891 (+ 5324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:06.598903 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:06.598907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:06.598950 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:06.602296 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:06.603452 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:06.603460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:06.603464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:06.603641 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:06.604207 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:06.604215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:06.604224 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:06.604550 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:06.604754 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:06.605654 (+ 900us) server_negotiation.cc:299] Negotiation successful
0807 22:44:06.605953 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":379,"threads_started":1}
I20260807 22:44:06.607626 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:06.583155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:43021)
0807 22:44:06.584575 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:06.584625 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:06.586700 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:06.587629 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:06.587638 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:06.588311 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:06.589358 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:06.589391 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.590749 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.590761 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:06.592628 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:06.592649 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.593697 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.593709 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:06.594023 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:06.595126 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:06.595193 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:06.598544 (+ 3351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:06.602528 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:06.602541 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:06.602573 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:06.603265 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:06.603786 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:06.603792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:06.603796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:06.604054 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:06.604722 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:06.604731 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:06.605187 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 22:44:06.605510 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 22:44:06.606201 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":350,"threads_started":1}
I20260807 22:44:06.609818 30071 heartbeater.cc:347] Connected to a master server at 127.20.10.190:42065
I20260807 22:44:06.610219 30071 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:06.611346 30071 heartbeater.cc:511] Master 127.20.10.190:42065 requested a full tablet report, sending...
I20260807 22:44:06.613274 29572 ts_manager.cc:194] Registered new tserver with Master: 6c73c4a58bc64cc2a061195f83d0915f (127.20.10.131:36131)
I20260807 22:44:06.614223 29572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:43021
I20260807 22:44:06.625279 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
Entry for principal oryx/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab.
WARNING: no policy specified for oryx/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
Entry for principal oryx/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab.
WARNING: no policy specified for oryx/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
Entry for principal oryx/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab.
I20260807 22:44:06.842523 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29639
Aug 07 22:44:06 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:44:06.864753 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:06.844110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.20.10.129:39409)
0807 22:44:06.845160 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:06.845174 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:06.845682 (+ 508us) server_negotiation.cc:407] Connection header received
0807 22:44:06.845928 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:06.845937 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:06.846194 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:06.846385 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:06.848325 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.849689 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.851228 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.851889 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.852061 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:06.857202 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:06.857220 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:06.857238 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:06.857285 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:06.860821 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:06.861796 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:06.861810 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:06.861817 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:06.862054 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:06.862606 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:06.862612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:06.862615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:06.863250 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:06.863542 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:06.863951 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 22:44:06.864186 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":502,"threads_started":1}
I20260807 22:44:06.866338 29771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:06.961225 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29639
I20260807 22:44:06.986287 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29784
I20260807 22:44:06.990137 29921 heartbeater.cc:503] Master 127.20.10.190:42065 was elected leader, sending a full tablet report...
Aug 07 22:44:06 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:44:07.005970 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:06.987646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.20.10.130:34229)
0807 22:44:06.988433 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:06.988441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:06.988477 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:44:06.988552 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:06.988558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:06.988831 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:06.988987 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:06.990399 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.991812 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.993250 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:06.993864 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:06.994029 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:06.999069 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:06.999087 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:06.999103 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:06.999148 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:07.002470 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:07.003493 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:07.003501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:07.003504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:07.003665 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:07.004111 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:07.004118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:07.004126 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:07.004529 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:07.004907 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:07.005237 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 22:44:07.005541 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":365,"threads_started":1}
I20260807 22:44:07.007580 29916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:07.102558 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29784
I20260807 22:44:07.127895 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29928
Aug 07 22:44:07 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:44:07.149823 30089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:07.129097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.20.10.131:36131)
0807 22:44:07.129811 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:07.129819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:07.129849 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:44:07.129916 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:07.129923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:07.130278 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:07.130437 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:07.132099 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:07.133384 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:07.135016 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:07.135699 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:07.135862 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:07.141449 (+ 5587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:07.141470 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:07.141488 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:07.141549 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:07.145736 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:07.146907 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:07.146917 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:07.146920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:07.147110 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:07.147613 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:07.147619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:07.147622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:07.148240 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:07.148521 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:07.149003 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 22:44:07.149428 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":306,"threads_started":1}
I20260807 22:44:07.151516 30066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:07.242511 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29928
I20260807 22:44:07.264665 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 29538
I20260807 22:44:07.266340 29604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:07.380861 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 29538
I20260807 22:44:07.409529 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42065
--webserver_interface=127.20.10.190
--webserver_port=37311
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:07.784421 30093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:07.784754 30093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:07.784937 30093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:07.792822 30093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:07.792958 30093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:07.793049 30093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:07.793167 30093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:07.793253 30093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:07.804036 30093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:42065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=37311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:07.805544 30093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:07.807101 30093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:07.820525 30099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:07.820708 30103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:07.821066 30100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:07.821955 30093 server_base.cc:1034] running on GCE node
I20260807 22:44:07.823021 30093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:07.824777 30093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:07.826017 30093 hybrid_clock.cc:648] HybridClock initialized: now 1786142647825951 us; error 54 us; skew 500 ppm
I20260807 22:44:07.826527 30093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:07 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142647, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:07.830209 30093 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:44:07.832183 30093 webserver.cc:460] Webserver started at http://127.20.10.190:37311/ using document root <none> and password file <none>
I20260807 22:44:07.833019 30093 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:07.833211 30093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:07.839758 30093 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:44:07.843214 30111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:07.844931 30093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 22:44:07.845194 30093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "36b31fbe3dc54e6cb416fe36f53f7c40"
format_stamp: "Formatted at 2026-08-07 22:44:04 on dist-test-slave-csg5"
server_key: "e8a3c378e4a3e47883e55b248d389c1b"
server_key_iv: "03c8225db1a41a41fe0236e1df9e6df1"
server_key_version: "encryptionkey@0"
I20260807 22:44:07.846421 30093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:07.865923 30093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:07.870828 30093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:07.871142 30093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:07.892520 30093 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:42065
I20260807 22:44:07.892541 30168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:42065 every 8 connection(s)
I20260807 22:44:07.894237 30093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:07.902413 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30093
I20260807 22:44:07.902843 30169 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:44:07.903596 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:39409
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:42065
--webserver_port=39031
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:44:07.909166 30169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Bootstrap starting.
I20260807 22:44:07.933205 30169 log.cc:824] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:07.943063 30169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:44:07.943679 30169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Bootstrap complete.
I20260807 22:44:07.954358 30169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:07.955495 30169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b31fbe3dc54e6cb416fe36f53f7c40, State: Initialized, Role: FOLLOWER
I20260807 22:44:07.956262 30169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:07.956535 30169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:07.956705 30169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:44:07.956912 30169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:44:07.961308 30169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:07.961935 30169 leader_election.cc:302] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36b31fbe3dc54e6cb416fe36f53f7c40; no voters:
I20260807 22:44:07.962702 30169 leader_election.cc:288] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:44:07.962865 30173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:44:07.964372 30173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [term 2 LEADER]: Becoming Leader. State: Replica: 36b31fbe3dc54e6cb416fe36f53f7c40, State: Running, Role: LEADER
I20260807 22:44:07.965188 30173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } }
I20260807 22:44:07.965399 30169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:07.968186 30177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b31fbe3dc54e6cb416fe36f53f7c40. Latest consensus state: current_term: 2 leader_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } } }
I20260807 22:44:07.968715 30177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:07.969445 30176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b31fbe3dc54e6cb416fe36f53f7c40" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42065 } } }
I20260807 22:44:07.969848 30176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:07.969753 30181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:07.986251 30181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:07.988317 30181 catalog_manager.cc:1259] Loaded cluster ID: 47119d9dad7b4a31a122cea7a3496244
I20260807 22:44:07.988466 30181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:07.993919 30181 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:07.995992 30181 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36b31fbe3dc54e6cb416fe36f53f7c40: Loaded TSK: 0
I20260807 22:44:07.997812 30181 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:44:08.320482 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:08.320859 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:08.321062 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:08.329962 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:08.330160 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:08.330384 30171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:44:08.343156 30171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.129.keytab
--principal=oryx/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:39409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=39031
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:08.344655 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:08.346089 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:08.366643 30198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:08.366783 30201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:08.367391 30199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:08.369496 30171 server_base.cc:1034] running on GCE node
I20260807 22:44:08.370287 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:08.371572 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:08.372871 30171 hybrid_clock.cc:648] HybridClock initialized: now 1786142648372790 us; error 60 us; skew 500 ppm
I20260807 22:44:08.373314 30171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:08 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142648, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:08.382395 30171 init.cc:377] Logged in from keytab as oryx/127.20.10.129@KRBTEST.COM (short username oryx)
I20260807 22:44:08.384524 30171 webserver.cc:460] Webserver started at http://127.20.10.129:39031/ using document root <none> and password file <none>
I20260807 22:44:08.385454 30171 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:08.385686 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:08.392812 30171 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:44:08.396595 30208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:08.398379 30171 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 22:44:08.398623 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "e38e595d75914e6ea12b4b1d2e01d726"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "b10cabf17462ed31114b36292936958b"
server_key_iv: "bba400d817d67b146ed3b6c232330c6a"
server_key_version: "encryptionkey@0"
I20260807 22:44:08.400278 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:08.417692 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:08.423884 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:08.424264 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:08.425690 30171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:08.427453 30171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:08.427606 30171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:08.427772 30171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:08.427879 30171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:08.469455 30171 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:39409
I20260807 22:44:08.469563 30328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:39409 every 8 connection(s)
I20260807 22:44:08.471216 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:08.472692 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30171
I20260807 22:44:08.473453 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:34229
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:42065
--webserver_port=34679
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:44:08 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:08 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:44:08.853606 30333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:08.853966 30333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:08.854130 30333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:08.861747 30333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:08.861928 30333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:08.862106 30333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:44:08.873474 30333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.130.keytab
--principal=oryx/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:34229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=34679
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:08.875123 30333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:08.876812 30333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:08.898643 30340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:08.901108 30339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:08.901520 30342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:08.905211 30333 server_base.cc:1034] running on GCE node
I20260807 22:44:08.906051 30333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:08.907410 30333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:08.908998 30333 hybrid_clock.cc:648] HybridClock initialized: now 1786142648908913 us; error 94 us; skew 500 ppm
I20260807 22:44:08.909432 30333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:08 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142648, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:08.913625 30333 init.cc:377] Logged in from keytab as oryx/127.20.10.130@KRBTEST.COM (short username oryx)
I20260807 22:44:08.916668 30333 webserver.cc:460] Webserver started at http://127.20.10.130:34679/ using document root <none> and password file <none>
I20260807 22:44:08.918219 30333 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:08.918542 30333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:08.930732 30333 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.003s
I20260807 22:44:08.936619 30349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:08.939615 30333 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260807 22:44:08.939981 30333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "21831f56c35846dda37ce616c39aee38"
format_stamp: "Formatted at 2026-08-07 22:44:05 on dist-test-slave-csg5"
server_key: "d525b9de842a82fed128cd4a0139499d"
server_key_iv: "7c3b17fc103841603897a3be0bcdb6f8"
server_key_version: "encryptionkey@0"
I20260807 22:44:08.942464 30333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:08.986778 30333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:08.992316 30333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:08.992632 30333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:08.993817 30333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:08.995471 30333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:08.995568 30333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:08.995724 30333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:08.995803 30333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:09.049666 30333 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:34229
I20260807 22:44:09.049715 30476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:34229 every 8 connection(s)
I20260807 22:44:09.051390 30333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:09.060339 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30333
I20260807 22:44:09.061144 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:36131
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:42065
--webserver_port=35147
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:44:09.841297 30481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:09.841810 30481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:09.842085 30481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:09.857959 30481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:09.858328 30481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:09.858608 30481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:44:09.880127 30481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/krb5kdc/oryx_127.20.10.131.keytab
--principal=oryx/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:36131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=35147
--tserver_master_addrs=127.20.10.190:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:09.882360 30481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:09.884306 30481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:09.913631 30332 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.423s user 0.004s sys 0.014s
W20260807 22:44:09.917155 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:08.475185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:33975 (local address 127.20.10.190:42065)
0807 22:44:08.476018 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:08.476038 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:08.476956 (+ 918us) server_negotiation.cc:407] Connection header received
0807 22:44:08.480274 (+ 3318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:08.480289 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:08.480679 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:08.481033 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:08.483407 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:08.484872 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:08.487711 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:08.488538 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:08.488795 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.915603 (+1426808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:33975: BlockingRecv error: recv got EOF from 127.20.10.129:33975 (error 108)
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":297,"threads_started":1}
I20260807 22:44:09.920356 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:08.475295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:33975)
0807 22:44:08.476715 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:08.476782 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:08.479617 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:08.481287 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:08.481300 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:08.482187 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:08.483069 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:08.483101 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:08.485236 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:08.485242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:08.487495 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:08.487514 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:08.488873 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:08.488883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:08.489214 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:08.490060 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:08.490093 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:09.918595 (+1428502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1089,"thread_start_us":871,"threads_started":1}
W20260807 22:44:09.922400 30329 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:42065 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.10.190:42065: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
W20260807 22:44:09.922508 30488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:09.923487 30487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:09.937057 30491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:09.939663 30481 server_base.cc:1034] running on GCE node
I20260807 22:44:09.940661 30481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:09.942296 30481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:09.945143 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:09.924485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:47589)
0807 22:44:09.927083 (+ 2598us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:09.927126 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:09.928271 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:09.929331 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:09.929339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:09.929944 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:09.930315 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.930337 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.932715 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.932724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.935300 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.935325 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.936826 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.936838 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.937185 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.938122 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:09.938179 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:09.944716 (+ 6537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2406,"thread_start_us":11325,"threads_started":1}
W20260807 22:44:09.945340 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:09.925202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:47589 (local address 127.20.10.190:42065)
0807 22:44:09.925529 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:09.925539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:09.927251 (+ 1712us) server_negotiation.cc:407] Connection header received
0807 22:44:09.928720 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:09.928730 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:09.928855 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:09.928960 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:09.930594 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.932401 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.935569 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.936257 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.936393 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.945029 (+ 8636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:47589: BlockingRecv error: recv got EOF from 127.20.10.129:47589 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 22:44:09.945789 30481 hybrid_clock.cc:648] HybridClock initialized: now 1786142649945578 us; error 165 us; skew 500 ppm
I20260807 22:44:09.946290 30481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142649, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:09.951653 30481 init.cc:377] Logged in from keytab as oryx/127.20.10.131@KRBTEST.COM (short username oryx)
I20260807 22:44:09.954171 30481 webserver.cc:460] Webserver started at http://127.20.10.131:35147/ using document root <none> and password file <none>
I20260807 22:44:09.955196 30481 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:09.955400 30481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:09 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:09.964274 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:09.948121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:42151)
0807 22:44:09.948384 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:09.948408 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:09.948572 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:09.949128 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:09.949134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:09.949662 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:09.949941 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.949960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.952520 (+ 2560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.952531 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.955206 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.955241 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.956871 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.956880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.957094 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.958055 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:09.958088 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:09.963898 (+ 5810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260807 22:44:09.964498 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:09.946344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:42151 (local address 127.20.10.190:42065)
0807 22:44:09.946604 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:09.946612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:09.948537 (+ 1925us) server_negotiation.cc:407] Connection header received
0807 22:44:09.948753 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:09.948762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:09.948903 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:09.949069 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:09.950173 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.952127 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.955548 (+ 3421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.956615 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.956825 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.964197 (+ 7372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:42151: BlockingRecv error: recv got EOF from 127.20.10.129:42151 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260807 22:44:09.965099 30329 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:44:09.965721 30481 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.008s
I20260807 22:44:09.969972 30499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:09.972115 30481 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 22:44:09.972395 30481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "6c73c4a58bc64cc2a061195f83d0915f"
format_stamp: "Formatted at 2026-08-07 22:44:06 on dist-test-slave-csg5"
server_key: "41bdbc08fdfac5f5b33cd27fb68db03f"
server_key_iv: "4db84b569d9acb5d0caa361db871ea95"
server_key_version: "encryptionkey@0"
I20260807 22:44:09.973841 30481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:10.036966 30481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:10.043484 30481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:10.043852 30481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:10.045181 30481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:10.047031 30481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:10.047196 30481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:10.047356 30481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:10.047457 30481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:10.118407 30481 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:36131
I20260807 22:44:10.118471 30622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:36131 every 8 connection(s)
I20260807 22:44:10.120054 30481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:10.124557 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30481
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:10.158946 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:10.125822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.20.10.190:42065)
0807 22:44:10.135811 (+ 9989us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:10.135821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:10.135860 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:44:10.135944 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:10.135952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:10.136069 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:10.136270 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:10.138245 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.139288 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.140708 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.141646 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.142041 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.147210 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:10.147229 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:10.147262 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:10.147335 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:10.153174 (+ 5839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:10.154768 (+ 1594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:10.154782 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:10.154794 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:10.155090 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:10.155713 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:10.155720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:10.155724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:10.156599 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:10.156978 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:10.157770 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 22:44:10.158294 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":249,"server-negotiator.queue_time_us":9776,"thread_start_us":9581,"threads_started":1}
W20260807 22:44:10.332948 30480 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.261s user 0.003s sys 0.015s
I20260807 22:44:10.334918 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:09.055732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:54057)
0807 22:44:09.056815 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:09.056857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:09.058890 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:09.059781 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:09.059795 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:09.060634 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:09.061540 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.061570 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.063383 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.063394 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.067413 (+ 4019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:09.067453 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.070007 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.070029 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:09.070593 (+ 564us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:09.071966 (+ 1373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:09.072031 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:10.333836 (+1261805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":491,"threads_started":1}
W20260807 22:44:10.334921 30479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:09.054728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:54057 (local address 127.20.10.190:42065)
0807 22:44:09.055247 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:09.055252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:09.056996 (+ 1744us) server_negotiation.cc:407] Connection header received
0807 22:44:09.059225 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:09.059231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:09.059336 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:09.059462 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:09.061802 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.063121 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.067955 (+ 4834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:09.069054 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:09.069264 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.334316 (+1265052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:54057: BlockingRecv error: recv got EOF from 127.20.10.130:54057 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":281,"threads_started":1}
W20260807 22:44:10.336304 30477 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:42065 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.10.190:42065: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:10.353554 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:10.337303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:59761)
0807 22:44:10.337650 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:10.337691 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:10.337893 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:10.338450 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:10.338458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:10.339305 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:10.339655 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.339686 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.341018 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.341028 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.342396 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.342411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.343576 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.343588 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.343925 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.344791 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:10.344831 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:10.353049 (+ 8218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260807 22:44:10.353730 30479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:10.337477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:59761 (local address 127.20.10.190:42065)
0807 22:44:10.337663 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:10.337667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:10.337815 (+ 148us) server_negotiation.cc:407] Connection header received
0807 22:44:10.338088 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:10.338092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:10.338261 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:10.338358 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:10.339907 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.340806 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.342551 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.343206 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.343365 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.353457 (+ 10092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:59761: BlockingRecv error: recv got EOF from 127.20.10.130:59761 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:10.372534 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:10.355506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:44663)
0807 22:44:10.355794 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:10.355816 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:10.355946 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:10.356779 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:10.356784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:10.357279 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:10.357526 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.357543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.359169 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.359175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.360746 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.360763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.361960 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.361965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.362161 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.366587 (+ 4426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:10.366616 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:10.372067 (+ 5451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156}
W20260807 22:44:10.372594 30479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:10.355876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44663 (local address 127.20.10.190:42065)
0807 22:44:10.356250 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:10.356256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:10.356276 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:44:10.356336 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:10.356342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:10.356449 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:10.356549 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:10.357771 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.358962 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.360926 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.361576 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.361684 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.372280 (+ 10596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:44663: BlockingRecv error: recv got EOF from 127.20.10.130:44663 (error 108)
Metrics: {"server-negotiator.queue_time_us":199}
W20260807 22:44:10.373091 30477 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:10 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:10.979719 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:10.966060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:58877)
0807 22:44:10.966924 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:10.966950 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:10.967104 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:10.967903 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:10.967909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:10.968621 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:10.968811 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.968823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.971078 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.971083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.972145 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.972162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.973701 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.973707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.973853 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.974609 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:10.974632 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:10.979431 (+ 4799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":417,"threads_started":1}
W20260807 22:44:10.980333 30631 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:10.966083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:58877 (local address 127.20.10.190:42065)
0807 22:44:10.967318 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:10.967329 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:10.967359 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:44:10.967447 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:10.967456 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:10.967588 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:10.967762 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:10.969038 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.970860 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.972354 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.973425 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.973615 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.979884 (+ 6269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:58877: BlockingRecv error: recv got EOF from 127.20.10.129:58877 (error 108)
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":810,"threads_started":1}
W20260807 22:44:11.311015 30626 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.168s user 0.004s sys 0.019s
W20260807 22:44:11.312400 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:10.123401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:57887 (local address 127.20.10.190:42065)
0807 22:44:10.123611 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:10.123616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:10.126077 (+ 2461us) server_negotiation.cc:407] Connection header received
0807 22:44:10.130486 (+ 4409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:10.130507 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:10.130649 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:10.130804 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:10.135710 (+ 4906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.136774 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.140811 (+ 4037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.141667 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.141837 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.312133 (+1170296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:57887: BlockingRecv error: recv got EOF from 127.20.10.131:57887 (error 108)
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":65}
I20260807 22:44:11.313100 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:10.124638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:57887)
0807 22:44:10.125842 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:10.125919 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:10.129912 (+ 3993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:10.131678 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:10.131695 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:10.132892 (+ 1197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:10.134123 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.134173 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.137069 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.137080 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.140605 (+ 3525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:10.140633 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:10.142009 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:10.142020 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:10.142353 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:10.143279 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:10.143314 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:11.311896 (+1168582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":1274,"threads_started":1}
W20260807 22:44:11.314374 30623 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:42065 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.10.190:42065: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:11.325973 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:11.315095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:49111)
0807 22:44:11.315337 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:11.315356 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:11.315473 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:11.315801 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:11.315805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:11.316191 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:11.316377 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.316386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.317572 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.317576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.318818 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.318834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.319971 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.319980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.320184 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.320994 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:11.321024 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:11.325704 (+ 4680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 22:44:11.326478 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:11.315219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:49111 (local address 127.20.10.190:42065)
0807 22:44:11.315485 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:11.315490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:11.315510 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:44:11.315566 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:11.315570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:11.315640 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:11.315727 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:11.316544 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.317414 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.319016 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.319709 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.319862 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.326073 (+ 6211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:49111: BlockingRecv error: recv got EOF from 127.20.10.131:49111 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:11.336720 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:11.326867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:54441)
0807 22:44:11.327052 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:11.327075 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:11.327195 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:11.327574 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:11.327577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:11.327943 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:11.328120 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.328128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.329362 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.329366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.330336 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.330347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.331501 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.331511 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.331740 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.332457 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:11.332477 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:11.336453 (+ 3976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:44:11.337087 30623 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:44:11.337232 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:11.326993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:54441 (local address 127.20.10.190:42065)
0807 22:44:11.327151 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:11.327156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:11.327177 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:44:11.327295 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:11.327300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:11.327389 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:11.327482 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:11.328322 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.329207 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.330492 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.331072 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.331280 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.336900 (+ 5620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:54441: BlockingRecv error: recv got EOF from 127.20.10.131:54441 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:11.385704 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:11.373934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:52059)
0807 22:44:11.374618 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:11.374637 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:11.374834 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:11.375387 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:11.375391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:11.375878 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:11.376042 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.376052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.377360 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.377365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.378352 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.378365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.379352 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.379358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.379530 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.380360 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:11.380381 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:11.385420 (+ 5039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":411,"threads_started":1}
W20260807 22:44:11.385986 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:11.373995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:52059 (local address 127.20.10.190:42065)
0807 22:44:11.374184 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:11.374188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:11.374798 (+ 610us) server_negotiation.cc:407] Connection header received
0807 22:44:11.374988 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:11.374993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:11.375061 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:11.375250 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:11.376254 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.377069 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.378596 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.379130 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.379275 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.385693 (+ 6418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:52059: BlockingRecv error: recv got EOF from 127.20.10.130:52059 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:11 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:11.993556 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:11.980992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:59911)
0807 22:44:11.981700 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:11.981718 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:11.981851 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:11.982404 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:11.982409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:11.983233 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:11.983469 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.983482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.984786 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.984791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.985876 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:11.985891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.986942 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.986947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:11.987083 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.987925 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:11.987973 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:11.993153 (+ 5180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":456,"threads_started":1}
W20260807 22:44:11.993835 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:11.981037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:59911 (local address 127.20.10.190:42065)
0807 22:44:11.981661 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:11.981665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:11.981849 (+ 184us) server_negotiation.cc:407] Connection header received
0807 22:44:11.981963 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:11.981967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:11.982042 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:11.982154 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:11.983659 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.984623 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.986096 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:11.986635 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:11.986880 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:11.993517 (+ 6637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:59911: BlockingRecv error: recv got EOF from 127.20.10.129:59911 (error 108)
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":374,"threads_started":1}
Aug 07 22:44:12 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:12 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:12.351614 30635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:12.338059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:36781)
0807 22:44:12.338732 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:12.338756 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:12.338910 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:12.339383 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:12.339387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:12.339852 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:12.340027 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.340037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.341271 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.341285 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:12.342346 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.342362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.343470 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.343480 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:12.343690 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:12.344434 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:12.344455 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:12.351143 (+ 6688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":385,"threads_started":1}
W20260807 22:44:12.351884 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:12.338179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:36781 (local address 127.20.10.190:42065)
0807 22:44:12.338562 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:12.338569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:12.338859 (+ 290us) server_negotiation.cc:407] Connection header received
0807 22:44:12.339028 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:12.339032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:12.339102 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:12.339198 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:12.340208 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.341032 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.342525 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.343235 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.343390 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:12.351539 (+ 8149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:36781: BlockingRecv error: recv got EOF from 127.20.10.131:36781 (error 108)
Metrics: {"server-negotiator.queue_time_us":199}
Aug 07 22:44:12 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:12 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:12.399235 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:12.387362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:58829)
0807 22:44:12.387935 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:12.387953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:12.388143 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:12.388560 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:12.388565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:12.389096 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:12.389306 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.389316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.390568 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.390574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:12.391746 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.391761 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.392770 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.392779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:12.392970 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:12.393676 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:12.393740 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:12.398795 (+ 5055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":317,"threads_started":1}
W20260807 22:44:12.399802 30634 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:12.387414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:58829 (local address 127.20.10.190:42065)
0807 22:44:12.387677 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:12.387685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:12.388053 (+ 368us) server_negotiation.cc:407] Connection header received
0807 22:44:12.388283 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:12.388289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:12.388401 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:12.388525 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:12.389479 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.390403 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.391926 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.392410 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.392667 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:12.399276 (+ 6609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:58829: BlockingRecv error: recv got EOF from 127.20.10.130:58829 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 22:44:12 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:13 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:13.005633 30637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:12.994715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:60941)
0807 22:44:12.995563 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:12.995586 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:12.995727 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:12.996134 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:12.996140 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:12.996647 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:12.996856 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.996868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.998049 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.998054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:12.999330 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:12.999348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.000192 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.000200 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:13.000362 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.001122 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:13.001147 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:13.005355 (+ 4208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":577,"threads_started":1}
W20260807 22:44:13.006064 30639 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:12.994853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:60941 (local address 127.20.10.190:42065)
0807 22:44:12.995601 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:12.995606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:12.995652 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:44:12.995869 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:12.995873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:12.995938 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:12.996028 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:12.997039 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.997869 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:12.999496 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:12.999995 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.000114 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.005713 (+ 5599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:60941: BlockingRecv error: recv got EOF from 127.20.10.129:60941 (error 108)
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":484,"threads_started":1}
Aug 07 22:44:13 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:13 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:13.369746 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:13.358389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:52183)
0807 22:44:13.359061 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:13.359090 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:13.359290 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:13.359671 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:13.359676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:13.360177 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:13.360373 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:13.360383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.361728 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.361732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:13.363232 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:13.363251 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.364169 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.364177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:13.364375 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.365065 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:13.365097 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:13.369333 (+ 4236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":393,"threads_started":1}
W20260807 22:44:13.369844 30639 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:13.358605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:52183 (local address 127.20.10.190:42065)
0807 22:44:13.358890 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:13.358922 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:13.359280 (+ 358us) server_negotiation.cc:407] Connection header received
0807 22:44:13.359422 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:13.359428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:13.359512 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:13.359620 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:13.360540 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.361518 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.363376 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.364003 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.364142 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.369573 (+ 5431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:52183: BlockingRecv error: recv got EOF from 127.20.10.131:52183 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 22:44:13 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:13 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:13.416656 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:13.400513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:55721)
0807 22:44:13.401227 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:13.401248 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:13.401456 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:13.401963 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:13.401968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:13.402484 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:13.402790 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:13.402802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.403986 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.403992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:13.405123 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:13.405138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.406220 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.406228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:13.406465 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.411178 (+ 4713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:13.411206 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:13.416308 (+ 5102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":394,"threads_started":1}
W20260807 22:44:13.416944 30639 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:13.400796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:55721 (local address 127.20.10.190:42065)
0807 22:44:13.401058 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:13.401064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:13.401370 (+ 306us) server_negotiation.cc:407] Connection header received
0807 22:44:13.401657 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:13.401662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:13.401786 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:13.401923 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:13.402967 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.403830 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.405428 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:13.406027 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:13.406196 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:13.416608 (+ 10412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:55721: BlockingRecv error: recv got EOF from 127.20.10.130:55721 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:14.018873 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:14.006832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:39731)
0807 22:44:14.007451 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:14.007471 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:14.007593 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:14.008273 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:14.008278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:14.008911 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:14.009152 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.009189 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.010802 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.010811 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.012004 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.012022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.012811 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.012816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.012952 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.013653 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:14.013675 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:14.018365 (+ 4690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":310,"threads_started":1}
W20260807 22:44:14.019095 30643 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:14.007075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:39731 (local address 127.20.10.190:42065)
0807 22:44:14.007902 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:14.007906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:14.007924 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:44:14.007986 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:14.007989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:14.008050 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:14.008145 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:14.009363 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.010532 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.012178 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.012620 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.012730 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.018793 (+ 6063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:39731: BlockingRecv error: recv got EOF from 127.20.10.129:39731 (error 108)
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":606,"threads_started":1}
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:14.381470 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:14.370977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:58159)
0807 22:44:14.371630 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:14.371651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:14.371826 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:14.372203 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:14.372207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:14.372687 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:14.372858 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.372867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.373939 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.373944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.374972 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.374985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.375704 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.375710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.375911 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.376629 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:14.376648 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:14.381076 (+ 4428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":399,"threads_started":1}
W20260807 22:44:14.381613 30643 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:14.371143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:58159 (local address 127.20.10.190:42065)
0807 22:44:14.371396 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:14.371404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:14.371794 (+ 390us) server_negotiation.cc:407] Connection header received
0807 22:44:14.371973 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:14.371979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:14.372061 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:14.372169 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:14.373004 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.373793 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.375124 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.375547 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.375688 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.381351 (+ 5663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:58159: BlockingRecv error: recv got EOF from 127.20.10.131:58159 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:14 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:14.429118 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:14.417937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:54631)
0807 22:44:14.418547 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:14.418574 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:14.418758 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:14.419206 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:14.419209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:14.419622 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:14.419830 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.419840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.421002 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.421006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.422089 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:14.422103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.423008 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.423013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:14.423158 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.423779 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:14.423841 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:14.428650 (+ 4809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":338,"threads_started":1}
W20260807 22:44:14.429173 30643 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:14.418090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:54631 (local address 127.20.10.190:42065)
0807 22:44:14.418277 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:14.418281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:14.418722 (+ 441us) server_negotiation.cc:407] Connection header received
0807 22:44:14.418956 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:14.418961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:14.419048 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:14.419142 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:14.419992 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.420862 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.422335 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:14.422831 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:14.422938 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:14.428902 (+ 5964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:54631: BlockingRecv error: recv got EOF from 127.20.10.130:54631 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:15.032002 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:15.020316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:60343)
0807 22:44:15.020963 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:15.020982 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:15.021127 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:15.021673 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:15.021677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:15.022309 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:15.022486 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.022497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.023953 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.023958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.025405 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.025422 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.026317 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.026322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.026468 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.027140 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:15.027159 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:15.031648 (+ 4489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":352,"threads_started":1}
W20260807 22:44:15.032305 30647 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:15.020507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:60343 (local address 127.20.10.190:42065)
0807 22:44:15.021084 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:15.021090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:15.021109 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:44:15.021315 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:15.021378 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:15.021474 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:15.021595 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:15.022722 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.023778 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.025606 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.026091 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.026206 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.031951 (+ 5745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:60343: BlockingRecv error: recv got EOF from 127.20.10.129:60343 (error 108)
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":349,"threads_started":1}
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:15.393883 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:15.382771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:33457)
0807 22:44:15.383394 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:15.383419 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:15.383614 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:15.384067 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:15.384074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:15.384673 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:15.384957 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.384971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.386141 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.386147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.387404 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.387419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.388168 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.388244 (+ 76us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.388439 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.389102 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:15.389204 (+ 102us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:15.393555 (+ 4351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":336,"threads_started":1}
W20260807 22:44:15.394135 30647 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:15.383028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:33457 (local address 127.20.10.190:42065)
0807 22:44:15.383248 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:15.383255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:15.383559 (+ 304us) server_negotiation.cc:407] Connection header received
0807 22:44:15.383769 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:15.383774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:15.383872 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:15.383987 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:15.385125 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.385979 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.387569 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.387992 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.388093 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.393830 (+ 5737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:33457: BlockingRecv error: recv got EOF from 127.20.10.131:33457 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:15 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:15.441833 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:15.430403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:45415)
0807 22:44:15.431033 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:15.431050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:15.431185 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:15.431593 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:15.431597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:15.432121 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:15.432370 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.432381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.433579 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.433584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.434873 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:15.434897 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.435823 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.435830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:15.436029 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.436831 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:15.436858 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:15.441490 (+ 4632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":350,"threads_started":1}
W20260807 22:44:15.442049 30647 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:15.430763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:45415 (local address 127.20.10.190:42065)
0807 22:44:15.430966 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:15.430971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:15.431135 (+ 164us) server_negotiation.cc:407] Connection header received
0807 22:44:15.431300 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:15.431304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:15.431397 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:15.431486 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:15.432580 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.433384 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.435099 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:15.435590 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:15.435767 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:15.441770 (+ 6003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:45415: BlockingRecv error: recv got EOF from 127.20.10.130:45415 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:16.044166 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:16.033249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:50141)
0807 22:44:16.033848 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:16.033871 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:16.034020 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:16.034530 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:16.034536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:16.035078 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:16.035247 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.035257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.036343 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.036349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.037362 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.037375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.038124 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.038130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.038353 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.039083 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:16.039104 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:16.043784 (+ 4680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":332,"threads_started":1}
W20260807 22:44:16.044718 30652 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:16.033331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:50141 (local address 127.20.10.190:42065)
0807 22:44:16.034231 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:16.034235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:16.034249 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:44:16.034294 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:16.034297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:16.034363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:16.034482 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:16.035411 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.036183 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.037521 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.037954 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.038095 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.044426 (+ 6331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:50141: BlockingRecv error: recv got EOF from 127.20.10.129:50141 (error 108)
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":631,"threads_started":1}
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:16.406183 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:16.395139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:57869)
0807 22:44:16.395735 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:16.395760 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:16.395925 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:16.396406 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:16.396410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:16.396931 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:16.397100 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.397110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.398371 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.398375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.399691 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.399708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.400923 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.400928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.401146 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.401965 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:16.401985 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:16.405939 (+ 3954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":324,"threads_started":1}
W20260807 22:44:16.406550 30652 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:16.395329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:57869 (local address 127.20.10.190:42065)
0807 22:44:16.395614 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:16.395621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:16.395907 (+ 286us) server_negotiation.cc:407] Connection header received
0807 22:44:16.396123 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:16.396127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:16.396201 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:16.396322 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:16.397290 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.398149 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.399922 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.400713 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.400881 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.406198 (+ 5317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:57869: BlockingRecv error: recv got EOF from 127.20.10.131:57869 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:16 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:16.454945 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:16.443068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:58225)
0807 22:44:16.443747 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:16.443771 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:16.443974 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:16.444391 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:16.444397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:16.444921 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:16.445131 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.445236 (+ 105us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.446513 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.446520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.447609 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:16.447621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.448480 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.448487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:16.448709 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.449632 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:16.449652 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:16.454625 (+ 4973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":377,"threads_started":1}
W20260807 22:44:16.455220 30652 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:16.443320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:58225 (local address 127.20.10.190:42065)
0807 22:44:16.443710 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:16.443719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:16.443881 (+ 162us) server_negotiation.cc:407] Connection header received
0807 22:44:16.444120 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:16.444127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:16.444230 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:16.444353 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:16.445413 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.446349 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.447777 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:16.448283 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:16.448446 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:16.454933 (+ 6487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:58225: BlockingRecv error: recv got EOF from 127.20.10.130:58225 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:17.056484 30656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:17.045472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:44379)
0807 22:44:17.046427 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:17.046445 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:17.046574 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:17.047186 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:17.047190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:17.047720 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:17.047990 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.048000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.049103 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.049107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.050269 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.050283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.051208 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.051215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.051403 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.052151 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:17.052183 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:17.056208 (+ 4025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":509,"threads_started":1}
W20260807 22:44:17.056937 30655 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:17.045593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:44379 (local address 127.20.10.190:42065)
0807 22:44:17.046102 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:17.046108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:17.046540 (+ 432us) server_negotiation.cc:407] Connection header received
0807 22:44:17.046860 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:17.046864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:17.046931 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:17.047027 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:17.048130 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.048951 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.050478 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.051023 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.051205 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.056441 (+ 5236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:44379: BlockingRecv error: recv got EOF from 127.20.10.129:44379 (error 108)
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:17.418874 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:17.407426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:45235)
0807 22:44:17.408277 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:17.408307 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:17.408505 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:17.409031 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:17.409036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:17.409612 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:17.409797 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.409807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.411068 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.411073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.412183 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.412197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.413116 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.413122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.413323 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.414036 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:17.414057 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:17.418499 (+ 4442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":609,"threads_started":1}
W20260807 22:44:17.419340 30655 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:17.407587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:45235 (local address 127.20.10.190:42065)
0807 22:44:17.407800 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:17.407808 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:17.408410 (+ 602us) server_negotiation.cc:407] Connection header received
0807 22:44:17.408757 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:17.408763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:17.408847 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:17.408968 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:17.409972 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.410885 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.412371 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.412906 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.413020 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.418903 (+ 5883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:45235: BlockingRecv error: recv got EOF from 127.20.10.131:45235 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:17 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:17.467258 30658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:17.456103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:49283)
0807 22:44:17.456805 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:17.456823 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:17.456988 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:17.457392 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:17.457395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:17.457908 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:17.458098 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.458108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.459348 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.459355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.460673 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:17.460690 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.461614 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.461620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:17.461810 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.462553 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:17.462572 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:17.466867 (+ 4295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":335,"threads_started":1}
W20260807 22:44:17.467407 30655 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:17.456324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49283 (local address 127.20.10.190:42065)
0807 22:44:17.456690 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:17.456697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:17.456927 (+ 230us) server_negotiation.cc:407] Connection header received
0807 22:44:17.457115 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:17.457119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:17.457209 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:17.457352 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:17.458343 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.459151 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.460846 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:17.461447 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:17.461587 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:17.467124 (+ 5537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:49283: BlockingRecv error: recv got EOF from 127.20.10.130:49283 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:18.069147 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:18.057575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:38133)
0807 22:44:18.058152 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:18.058170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:18.058286 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:18.058964 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:18.058968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:18.059485 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:18.059670 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.059681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.061368 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.061373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.062346 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.062359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.063406 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.063411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.063535 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.064184 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:18.064203 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:18.068750 (+ 4547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":291,"threads_started":1}
W20260807 22:44:18.069288 30660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:18.057716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:38133 (local address 127.20.10.190:42065)
0807 22:44:18.058515 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:18.058521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:18.058541 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:44:18.058608 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:18.058614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:18.058775 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:18.058889 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:18.059830 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.061058 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.062505 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.063151 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.063333 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.069016 (+ 5683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:38133: BlockingRecv error: recv got EOF from 127.20.10.129:38133 (error 108)
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":505,"threads_started":1}
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:18.434551 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:18.420172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:35305)
0807 22:44:18.421004 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:18.421027 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:18.421195 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:18.421612 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:18.421617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:18.422168 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:18.422413 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.422427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.423678 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.423683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.424794 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.424852 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.425787 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.425792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.425968 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.430216 (+ 4248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:18.430249 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:18.434299 (+ 4050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":334,"threads_started":1}
W20260807 22:44:18.435060 30660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:18.420413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:35305 (local address 127.20.10.190:42065)
0807 22:44:18.420644 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:18.420649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:18.421231 (+ 582us) server_negotiation.cc:407] Connection header received
0807 22:44:18.421309 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:18.421314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:18.421390 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:18.421498 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:18.422625 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.423521 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.425034 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.425605 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.425719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.434759 (+ 9040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:35305: BlockingRecv error: recv got EOF from 127.20.10.131:35305 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:18 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:18.481279 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:18.468559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:41981)
0807 22:44:18.469267 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:18.469286 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:18.469421 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:18.469803 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:18.469808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:18.470287 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:18.470622 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.470638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.472141 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.472149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.473857 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:18.473876 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.474864 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.474870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:18.475013 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.475904 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:18.475925 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:18.480864 (+ 4939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":371,"threads_started":1}
W20260807 22:44:18.481714 30660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:18.468687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:41981 (local address 127.20.10.190:42065)
0807 22:44:18.468964 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:18.468971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:18.469417 (+ 446us) server_negotiation.cc:407] Connection header received
0807 22:44:18.469539 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:18.469543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:18.469624 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:18.469711 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:18.470999 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.471948 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.474077 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:18.474583 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:18.474762 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:18.481111 (+ 6349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:41981: BlockingRecv error: recv got EOF from 127.20.10.130:41981 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:19.081398 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:19.070487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:48467)
0807 22:44:19.071303 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:19.071336 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:19.071499 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:19.071941 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:19.071944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:19.072449 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:19.072621 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.072672 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.073791 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.073796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.074974 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.074989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.075777 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.075782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.075934 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.076600 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:19.076620 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:19.081110 (+ 4490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":350,"threads_started":1}
W20260807 22:44:19.081787 30664 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:19.070522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48467 (local address 127.20.10.190:42065)
0807 22:44:19.071630 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:19.071636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:19.071657 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:44:19.071708 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:19.071712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:19.071776 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:19.071873 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:19.072839 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.073631 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.075141 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.075596 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.075697 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.081434 (+ 5737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:48467: BlockingRecv error: recv got EOF from 127.20.10.129:48467 (error 108)
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":854,"threads_started":1}
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:19.447372 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:19.435860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:49395)
0807 22:44:19.436678 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:19.436712 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:19.436871 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:19.437422 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:19.437431 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:19.437966 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:19.438237 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.438249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.439448 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.439453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.440581 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.440597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.441555 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.441561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.441704 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.442524 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:19.442547 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:19.447068 (+ 4521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":426,"threads_started":1}
W20260807 22:44:19.447618 30664 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:19.436080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:49395 (local address 127.20.10.190:42065)
0807 22:44:19.436424 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:19.436433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:19.436856 (+ 423us) server_negotiation.cc:407] Connection header received
0807 22:44:19.437008 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:19.437015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:19.437109 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:19.437209 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:19.438399 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.439250 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.440832 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.441330 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.441446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.447333 (+ 5887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:49395: BlockingRecv error: recv got EOF from 127.20.10.131:49395 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:19 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:19.494735 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:19.482702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:52793)
0807 22:44:19.483406 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:19.483425 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:19.483565 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:19.483966 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:19.483970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:19.484504 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:19.484674 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.484684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.486076 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.486081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.487391 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:19.487408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.488466 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.488473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:19.488683 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.489627 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:19.489659 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:19.494309 (+ 4650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":391,"threads_started":1}
W20260807 22:44:19.494786 30664 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:19.482787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:52793 (local address 127.20.10.190:42065)
0807 22:44:19.483125 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:19.483131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:19.483573 (+ 442us) server_negotiation.cc:407] Connection header received
0807 22:44:19.483675 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:19.483680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:19.483781 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:19.483884 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:19.484889 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.485904 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.487611 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:19.488267 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:19.488435 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:19.494427 (+ 5992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:52793: BlockingRecv error: recv got EOF from 127.20.10.130:52793 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:20.094401 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:20.082828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:34597)
0807 22:44:20.084027 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:20.084045 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:20.084188 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:20.084584 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:20.084588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:20.084986 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:20.085177 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.085187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.086426 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.086430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.087577 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.087590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.088403 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.088407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.088530 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.089200 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:20.089228 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:20.094052 (+ 4824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":963,"threads_started":1}
W20260807 22:44:20.094578 30667 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:20.082846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:34597 (local address 127.20.10.190:42065)
0807 22:44:20.083640 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:20.083650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:20.084228 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:44:20.084302 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:20.084306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:20.084389 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:20.084500 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:20.085349 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.086269 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.087729 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.088212 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.088311 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.094340 (+ 6029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:34597: BlockingRecv error: recv got EOF from 127.20.10.129:34597 (error 108)
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":459,"threads_started":1}
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:20.460532 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:20.448621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:46809)
0807 22:44:20.449367 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:20.449391 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:20.449514 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:20.450010 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:20.450015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:20.450518 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:20.450716 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.450726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.452048 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.452053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.453324 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.453341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.454258 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.454263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.454440 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.455246 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:20.455266 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:20.460264 (+ 4998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":397,"threads_started":1}
W20260807 22:44:20.460779 30667 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:20.448704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:46809 (local address 127.20.10.190:42065)
0807 22:44:20.448899 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:20.448904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:20.449490 (+ 586us) server_negotiation.cc:407] Connection header received
0807 22:44:20.449620 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:20.449624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:20.449782 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:20.449875 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:20.450899 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.451895 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.453529 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.454056 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.454172 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.460459 (+ 6287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:46809: BlockingRecv error: recv got EOF from 127.20.10.131:46809 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:20 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:20.508072 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:20.496134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:57807)
0807 22:44:20.496795 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:20.496819 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:20.496996 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:20.497655 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:20.497660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:20.498282 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:20.498481 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.498492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.499938 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.499943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.501211 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:20.501226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.502048 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.502059 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:20.502302 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.503030 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:20.503052 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:20.507771 (+ 4719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":339,"threads_started":1}
W20260807 22:44:20.508369 30667 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:20.496428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:57807 (local address 127.20.10.190:42065)
0807 22:44:20.496615 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:20.496621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:20.496973 (+ 352us) server_negotiation.cc:407] Connection header received
0807 22:44:20.497346 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:20.497356 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:20.497464 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:20.497559 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:20.498639 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.499728 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.501376 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:20.501817 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:20.501968 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:20.508064 (+ 6096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:57807: BlockingRecv error: recv got EOF from 127.20.10.130:57807 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:21.108091 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:21.095600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:59043)
0807 22:44:21.097377 (+ 1777us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:21.097394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:21.097524 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:21.098061 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:21.098065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:21.098542 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:21.098780 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.098791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.100165 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.100172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.101158 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.101173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.102008 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.102015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.102228 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.103036 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:21.103058 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:21.107745 (+ 4687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1645,"thread_start_us":1515,"threads_started":1}
W20260807 22:44:21.108894 30672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:21.095800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:59043 (local address 127.20.10.190:42065)
0807 22:44:21.097296 (+ 1496us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:21.097307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:21.097506 (+ 199us) server_negotiation.cc:407] Connection header received
0807 22:44:21.097652 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:21.097656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:21.097736 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:21.098026 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:21.098960 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.099962 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.101340 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.101823 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.101973 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.108193 (+ 6220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:59043: BlockingRecv error: recv got EOF from 127.20.10.129:59043 (error 108)
Metrics: {"server-negotiator.queue_time_us":1230,"thread_start_us":1001,"threads_started":1}
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:21.471947 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:21.461789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:40119)
0807 22:44:21.462366 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:21.462388 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:21.462563 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:21.462957 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:21.462962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:21.463369 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:21.463543 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.463552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.464611 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.464619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.465813 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.465827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.466705 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.466712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.466897 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.467523 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:21.467540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:21.471542 (+ 4002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":299,"threads_started":1}
W20260807 22:44:21.472069 30672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:21.461857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:40119 (local address 127.20.10.190:42065)
0807 22:44:21.462106 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:21.462115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:21.462488 (+ 373us) server_negotiation.cc:407] Connection header received
0807 22:44:21.462747 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:21.462752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:21.462817 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:21.462922 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:21.463698 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.464465 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.465969 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.466524 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.466657 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.471827 (+ 5170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:40119: BlockingRecv error: recv got EOF from 127.20.10.131:40119 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:21 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:21.520553 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:21.509525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:51947)
0807 22:44:21.510160 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:21.510176 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:21.510310 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:21.510761 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:21.510766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:21.511297 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:21.511515 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.511526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.512716 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.512723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.513847 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:21.513862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.514723 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.514731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:21.514936 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.515629 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:21.515656 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:21.520198 (+ 4542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":335,"threads_started":1}
W20260807 22:44:21.520740 30672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:21.509854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:51947 (local address 127.20.10.190:42065)
0807 22:44:21.510081 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:21.510088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:21.510285 (+ 197us) server_negotiation.cc:407] Connection header received
0807 22:44:21.510447 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:21.510452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:21.510553 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:21.510727 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:21.511714 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.512545 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.514027 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.514496 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.514682 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.520454 (+ 5772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:51947: BlockingRecv error: recv got EOF from 127.20.10.130:51947 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:22.121482 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.109759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:52597)
0807 22:44:22.110403 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:22.110422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:22.110587 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:22.111374 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:22.111380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:22.111997 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:22.112290 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.112302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.113689 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.113694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.114855 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.114871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.115777 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.115782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.115934 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.116798 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:22.116821 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:22.121157 (+ 4336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":344,"threads_started":1}
W20260807 22:44:22.122273 30677 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:22.109907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:52597 (local address 127.20.10.190:42065)
0807 22:44:22.110459 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:22.110464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:22.110490 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:44:22.110893 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:22.110901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:22.111010 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:22.111138 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:22.112474 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.113486 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.115051 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.115591 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.115779 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.121977 (+ 6198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:52597: BlockingRecv error: recv got EOF from 127.20.10.129:52597 (error 108)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":315,"threads_started":1}
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:22.484633 30678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.473293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:54223)
0807 22:44:22.473819 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:22.473841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:22.473968 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:22.474340 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:22.474344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:22.474907 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:22.475115 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.475126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.476485 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.476490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.477757 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.477777 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.479120 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.479136 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.479519 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.480299 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:22.480318 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:22.484384 (+ 4066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":384,"threads_started":1}
W20260807 22:44:22.484833 30677 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:22.473458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:54223 (local address 127.20.10.190:42065)
0807 22:44:22.473679 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:22.473686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:22.473951 (+ 265us) server_negotiation.cc:407] Connection header received
0807 22:44:22.474077 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:22.474083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:22.474181 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:22.474309 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:22.475411 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.476270 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.478066 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.478888 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.479087 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.484624 (+ 5537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:54223: BlockingRecv error: recv got EOF from 127.20.10.131:54223 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:22 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:22.533874 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.521781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:35665)
0807 22:44:22.522553 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:22.522573 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:22.522849 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:22.523275 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:22.523280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:22.523866 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:22.524057 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.524068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.525704 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.525710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.526890 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.526905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.527782 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.527789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.527997 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.528746 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:22.528769 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:22.533518 (+ 4749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":411,"threads_started":1}
W20260807 22:44:22.534283 30677 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:22.522005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:35665 (local address 127.20.10.190:42065)
0807 22:44:22.522413 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:22.522423 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:22.522723 (+ 300us) server_negotiation.cc:407] Connection header received
0807 22:44:22.523005 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:22.523011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:22.523114 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:22.523237 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:22.524232 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.525516 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.527057 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.527556 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.527718 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.533955 (+ 6237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:35665: BlockingRecv error: recv got EOF from 127.20.10.130:35665 (error 108)
Metrics: {"server-negotiator.queue_time_us":241}
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:23.134214 30680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.122734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:35977)
0807 22:44:23.123388 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.123420 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.123560 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.123979 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.123983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.124559 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:23.124734 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.124744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.125911 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.125915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.127031 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.127045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.127998 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.128002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.128144 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.128893 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:23.128918 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:23.133792 (+ 4874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":314,"threads_started":1}
W20260807 22:44:23.134479 30681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:23.122855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:35977 (local address 127.20.10.190:42065)
0807 22:44:23.123365 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.123370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.123570 (+ 200us) server_negotiation.cc:407] Connection header received
0807 22:44:23.123678 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.123682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.123785 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.123882 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:23.124916 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.125738 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.127228 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.127834 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.127954 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.134145 (+ 6191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:35977: BlockingRecv error: recv got EOF from 127.20.10.129:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":313,"threads_started":1}
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:23.499881 30682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.486013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:47039)
0807 22:44:23.486622 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.486644 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.486792 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.487164 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.487167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.487637 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:23.487815 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.487826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.489111 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.489115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.490187 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.490210 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.491240 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.491247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.491426 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.495522 (+ 4096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:23.495549 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:23.499503 (+ 3954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":345,"threads_started":1}
W20260807 22:44:23.500150 30681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:23.486148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:47039 (local address 127.20.10.190:42065)
0807 22:44:23.486416 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.486420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.486781 (+ 361us) server_negotiation.cc:407] Connection header received
0807 22:44:23.486925 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.486929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.486998 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.487087 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:23.487994 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.488935 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.490381 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.490943 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.491056 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.499902 (+ 8846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:47039: BlockingRecv error: recv got EOF from 127.20.10.131:47039 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:23 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:23.545691 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.535053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:35145)
0807 22:44:23.535639 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.535657 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.535838 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.536383 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.536388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.536877 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:23.537036 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.537045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.538257 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.538262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.539467 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.539482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.540350 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.540355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.540513 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.541387 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:23.541408 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:23.545426 (+ 4018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":344,"threads_started":1}
W20260807 22:44:23.546089 30681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:23.535242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:35145 (local address 127.20.10.190:42065)
0807 22:44:23.535465 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.535473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.535733 (+ 260us) server_negotiation.cc:407] Connection header received
0807 22:44:23.536031 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.536036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.536125 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.536319 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:23.537175 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.538088 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.539634 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.540092 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.540198 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.545725 (+ 5527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:35145: BlockingRecv error: recv got EOF from 127.20.10.130:35145 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.129@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:24.146950 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:24.135904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.129:48889)
0807 22:44:24.136490 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:24.136508 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:24.136685 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:24.137117 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:24.137121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:24.137539 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:24.137781 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.137825 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.139112 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.139117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.140141 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.140154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.140933 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.140938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.141090 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.141853 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:24.141874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:24.146563 (+ 4689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":312,"threads_started":1}
W20260807 22:44:24.147295 30688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:24.136108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48889 (local address 127.20.10.190:42065)
0807 22:44:24.136816 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:24.136820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:24.136839 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:44:24.136895 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:24.136899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:24.136961 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:24.137068 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:24.138011 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.138941 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.140302 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.140770 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.140869 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.146921 (+ 6052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:48889: BlockingRecv error: recv got EOF from 127.20.10.129:48889 (error 108)
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":464,"threads_started":1}
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.131@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:24.513180 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:24.501216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.131:37739)
0807 22:44:24.501826 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:24.501856 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:24.502002 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:24.502455 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:24.502458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:24.503004 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:24.503172 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.503182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.504548 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.504553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.506032 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.506050 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.507118 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.507135 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.507350 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.508119 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:24.508144 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:24.512778 (+ 4634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
W20260807 22:44:24.513305 30688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:24.501371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:37739 (local address 127.20.10.190:42065)
0807 22:44:24.501673 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:24.501678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:24.501979 (+ 301us) server_negotiation.cc:407] Connection header received
0807 22:44:24.502127 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:24.502131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:24.502221 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:24.502344 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:24.503380 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.504294 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.506262 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.506942 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.507100 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.513029 (+ 5929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.131:37739: BlockingRecv error: recv got EOF from 127.20.10.131:37739 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:44:24 dist-test-slave-csg5 krb5kdc[29523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.10.130@KRBTEST.COM for oryx/127.20.10.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:44:24.559207 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:24.547254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:42065 (local address 127.20.10.130:34429)
0807 22:44:24.547803 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:24.547820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:24.547948 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:24.548394 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:24.548398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:24.548814 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:24.549021 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.549031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.550287 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.550292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.551687 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:24.551706 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.552660 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.552666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:24.552852 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.553871 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:24.553894 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:24.558754 (+ 4860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.10.190:42065: Server oryx/127.20.10.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":279,"threads_started":1}
W20260807 22:44:24.559360 30688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:24.547299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:34429 (local address 127.20.10.190:42065)
0807 22:44:24.547581 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:24.547588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:24.547933 (+ 345us) server_negotiation.cc:407] Connection header received
0807 22:44:24.548098 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:24.548103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:24.548194 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:24.548299 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:24.549272 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.550117 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.551911 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:24.552464 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:24.552584 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:24.559053 (+ 6469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.130:34429: BlockingRecv error: recv got EOF from 127.20.10.130:34429 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:44:25.136099 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30171
I20260807 22:44:25.153393 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:25.137608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.20.10.129:39409)
0807 22:44:25.138403 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:25.138411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:25.138441 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:44:25.138520 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:25.138528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:25.138929 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:25.139120 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:25.140857 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.142133 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.143596 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.144297 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.144426 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:25.148030 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:25.148050 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:25.148076 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:25.148146 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:25.152656 (+ 4510us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:44:25.153079 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33178: Request ticket server kudu/127.20.10.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":455,"threads_started":1}
W20260807 22:44:25.153532 30692 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33178: Request ticket server kudu/127.20.10.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:44:25.153975 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30171: Not authorized: Client connection negotiation failed: client connection to 127.20.10.129:39409: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:44:25.154203 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30171
I20260807 22:44:25.180444 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30333
W20260807 22:44:25.180802 30694 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:25.148378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:56419 (local address 127.20.10.190:42065)
0807 22:44:25.148948 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:25.148952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:25.149522 (+ 570us) server_negotiation.cc:407] Connection header received
0807 22:44:25.149693 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:25.149701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:25.149796 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:25.149921 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:25.151478 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.152713 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.180223 (+ 27510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.10.129:56419: BlockingRecv error: recv got EOF from 127.20.10.129:56419 (error 108)
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":314,"threads_started":1}
I20260807 22:44:25.195782 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:25.182067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.20.10.130:34229)
0807 22:44:25.182781 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:25.182792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:25.182824 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:44:25.182911 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:25.182920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:25.183751 (+ 831us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:25.184005 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:25.185382 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.186590 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.188292 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.188897 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.189013 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:25.191962 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:25.191972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:25.191987 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:25.192031 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:25.195204 (+ 3173us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:44:25.195542 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47794: Request ticket server kudu/127.20.10.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":295,"threads_started":1}
W20260807 22:44:25.195937 30696 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47794: Request ticket server kudu/127.20.10.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:44:25.196027 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30333: Not authorized: Client connection negotiation failed: client connection to 127.20.10.130:34229: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:44:25.196260 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30333
I20260807 22:44:25.221944 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30481
I20260807 22:44:25.237506 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:25.223442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.20.10.131:36131)
0807 22:44:25.224157 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:25.224165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:25.224299 (+ 134us) server_negotiation.cc:407] Connection header received
0807 22:44:25.224381 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:25.224388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:25.224673 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:25.224924 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:25.226589 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.227807 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.229127 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.229757 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.229912 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:25.233114 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:25.233125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:25.233140 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:25.233185 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:25.236710 (+ 3525us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 22:44:25.237086 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39226: Request ticket server kudu/127.20.10.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":363,"threads_started":1}
W20260807 22:44:25.237637 30697 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39226: Request ticket server kudu/127.20.10.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 22:44:25.237845 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30481: Not authorized: Client connection negotiation failed: client connection to 127.20.10.131:36131: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.10.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 22:44:25.238037 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30481
I20260807 22:44:25.261062 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30093
I20260807 22:44:25.262512 30164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:25.393426 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30093
2026-08-07T22:44:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22939 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T22:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:25Z Disabled control of system clock
I20260807 22:44:25.502363 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46167
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:44:25.876456 30703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:25.876798 30703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:25.876964 30703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:25.884732 30703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:25.884869 30703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:25.884938 30703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:25.885030 30703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:25.885110 30703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:25.895045 30703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:25.896528 30703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:25.898475 30703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:25.912393 30708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:25.912427 30709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:25.912357 30711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:25.912817 30703 server_base.cc:1034] running on GCE node
I20260807 22:44:25.913740 30703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:25.915489 30703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:25.916792 30703 hybrid_clock.cc:648] HybridClock initialized: now 1786142665916727 us; error 56 us; skew 500 ppm
I20260807 22:44:25.917313 30703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:25.919999 30703 webserver.cc:460] Webserver started at http://127.20.10.190:36137/ using document root <none> and password file <none>
I20260807 22:44:25.920800 30703 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:25.920938 30703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:25.921295 30703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:25.924625 30703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6a62c890aed2457b89f724b82fb5c419"
format_stamp: "Formatted at 2026-08-07 22:44:25 on dist-test-slave-csg5"
server_key: "0560f6adbed18be1df6f9035cb1d4147"
server_key_iv: "5fa67616728b38609af2976c97f39416"
server_key_version: "encryptionkey@0"
I20260807 22:44:25.925549 30703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6a62c890aed2457b89f724b82fb5c419"
format_stamp: "Formatted at 2026-08-07 22:44:25 on dist-test-slave-csg5"
server_key: "0560f6adbed18be1df6f9035cb1d4147"
server_key_iv: "5fa67616728b38609af2976c97f39416"
server_key_version: "encryptionkey@0"
I20260807 22:44:25.931643 30703 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 22:44:25.936053 30719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:25.937731 30703 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:44:25.937964 30703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6a62c890aed2457b89f724b82fb5c419"
format_stamp: "Formatted at 2026-08-07 22:44:25 on dist-test-slave-csg5"
server_key: "0560f6adbed18be1df6f9035cb1d4147"
server_key_iv: "5fa67616728b38609af2976c97f39416"
server_key_version: "encryptionkey@0"
I20260807 22:44:25.938232 30703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:25.955770 30703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:25.956826 30703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:25.957155 30703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:25.980093 30703 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46167
I20260807 22:44:25.980291 30776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46167 every 8 connection(s)
I20260807 22:44:25.981627 30703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:25.986608 30777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:25.986591 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30703
I20260807 22:44:25.986896 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:44:25.987890 20522 external_mini_cluster.cc:1428] Setting key 2f4adc8794fba1cbf545ba1fe1376b6d
I20260807 22:44:26.003674 30777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419: Bootstrap starting.
I20260807 22:44:26.005358 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:25.990449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.20.10.190:46167)
0807 22:44:25.991745 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:25.991758 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:25.991806 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:44:25.994057 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:25.994086 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:25.994847 (+ 761us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:25.995251 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:25.996113 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:25.997437 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:25.999824 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:26.000745 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:26.000899 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:26.002398 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:26.002407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:26.002424 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:26.003109 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:26.003202 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:26.003227 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:26.003331 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:26.003679 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:44:26.004132 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":515,"threads_started":1}
I20260807 22:44:26.007966 30777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:26.009361 30777 log.cc:824] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:26.012759 30777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419: No bootstrap required, opened a new log
I20260807 22:44:26.022966 30777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } }
I20260807 22:44:26.023438 30777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:26.023561 30777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a62c890aed2457b89f724b82fb5c419, State: Initialized, Role: FOLLOWER
I20260807 22:44:26.024401 30777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } }
I20260807 22:44:26.024771 30777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:26.024988 30777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:44:26.025211 30777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:26.028586 30777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } }
I20260807 22:44:26.029425 30777 leader_election.cc:302] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a62c890aed2457b89f724b82fb5c419; no voters:
I20260807 22:44:26.030324 30777 leader_election.cc:288] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:44:26.030565 30782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:26.031445 30782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [term 1 LEADER]: Becoming Leader. State: Replica: 6a62c890aed2457b89f724b82fb5c419, State: Running, Role: LEADER
I20260807 22:44:26.032035 30782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } }
I20260807 22:44:26.032846 30777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:26.035081 30784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a62c890aed2457b89f724b82fb5c419. Latest consensus state: current_term: 1 leader_uuid: "6a62c890aed2457b89f724b82fb5c419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } } }
I20260807 22:44:26.035241 30783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a62c890aed2457b89f724b82fb5c419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62c890aed2457b89f724b82fb5c419" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46167 } } }
I20260807 22:44:26.035699 30784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:26.035701 30783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:26.038378 30788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:26.044307 30788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:26.055284 30788 catalog_manager.cc:1347] Generated new cluster ID: 2e41cd76f3074d648bb342743b8ff4ad
I20260807 22:44:26.055450 30788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:26.078523 30788 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:44:26.080016 30788 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:26.087785 30788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a62c890aed2457b89f724b82fb5c419: Generated new TSK 0
I20260807 22:44:26.088806 30788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:44:26.094944 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--builtin_ntp_servers=127.20.10.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 22:44:26.501626 30804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:26.502008 30804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:26.502105 30804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:26.510421 30804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:26.510560 30804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:26.510741 30804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:44:26.520781 30804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:26.522197 30804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:26.524417 30804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:26.540323 30811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:26.540349 30810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:26.540372 30813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:26.541245 30804 server_base.cc:1034] running on GCE node
I20260807 22:44:26.542048 30804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:26.543388 30804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:26.544761 30804 hybrid_clock.cc:648] HybridClock initialized: now 1786142666544638 us; error 98 us; skew 500 ppm
I20260807 22:44:26.545230 30804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:26.548427 30804 webserver.cc:460] Webserver started at http://127.20.10.129:39571/ using document root <none> and password file <none>
I20260807 22:44:26.549281 30804 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:26.549448 30804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:26.549830 30804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:26.553279 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "c5750b4a46fa43549d23b89c9ba12114"
format_stamp: "Formatted at 2026-08-07 22:44:26 on dist-test-slave-csg5"
server_key: "ed9f75df641627251ac6b7c9baef62a8"
server_key_iv: "66c4f2302abed292e1f61b90aa94fb70"
server_key_version: "encryptionkey@0"
I20260807 22:44:26.554214 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "c5750b4a46fa43549d23b89c9ba12114"
format_stamp: "Formatted at 2026-08-07 22:44:26 on dist-test-slave-csg5"
server_key: "ed9f75df641627251ac6b7c9baef62a8"
server_key_iv: "66c4f2302abed292e1f61b90aa94fb70"
server_key_version: "encryptionkey@0"
I20260807 22:44:26.560400 30804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 22:44:26.564841 30819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:26.566748 30804 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 22:44:26.567024 30804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "c5750b4a46fa43549d23b89c9ba12114"
format_stamp: "Formatted at 2026-08-07 22:44:26 on dist-test-slave-csg5"
server_key: "ed9f75df641627251ac6b7c9baef62a8"
server_key_iv: "66c4f2302abed292e1f61b90aa94fb70"
server_key_version: "encryptionkey@0"
I20260807 22:44:26.567314 30804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:26.591768 30804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:26.592835 30804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:26.593225 30804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:26.594461 30804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:26.596529 30804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:26.596690 30804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:26.596832 30804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:26.596920 30804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:26.638872 30804 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41187
I20260807 22:44:26.638988 30950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41187 every 8 connection(s)
I20260807 22:44:26.640380 30804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:26.645150 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30804
I20260807 22:44:26.645510 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:44:26.646247 20522 external_mini_cluster.cc:1428] Setting key c7b55ff54e3c0d0f30ec9de390c54882
I20260807 22:44:26.649670 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--builtin_ntp_servers=127.20.10.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:44:26.657874 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:26.643707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:46493 (local address 127.20.10.190:46167)
0807 22:44:26.644191 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:26.644196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:26.645899 (+ 1703us) server_negotiation.cc:407] Connection header received
0807 22:44:26.648500 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:26.648504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:26.648575 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:26.648733 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:26.650054 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:26.651422 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:26.653582 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:26.654384 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:26.654549 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:26.656796 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:26.656811 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:26.656818 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:26.656847 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:26.656903 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:26.656921 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:26.657011 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:26.657320 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:44:26.657443 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260807 22:44:26.659696 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:26.644421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46167 (local address 127.20.10.129:46493)
0807 22:44:26.645723 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:26.645775 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:26.648107 (+ 2332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:26.648963 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:26.648972 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:26.649070 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:26.649831 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:26.649867 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:26.651627 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:26.651632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:26.653366 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:26.653389 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:26.654952 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:26.654964 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:26.655370 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:26.656321 (+ 951us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:26.656327 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:26.656397 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:26.656534 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:26.657035 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:26.657052 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:26.657065 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:44:26.657300 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 22:44:26.657842 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":404,"threads_started":1}
I20260807 22:44:26.661463 30951 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46167
I20260807 22:44:26.661795 30951 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:26.662732 30951 heartbeater.cc:511] Master 127.20.10.190:46167 requested a full tablet report, sending...
I20260807 22:44:26.665480 30737 ts_manager.cc:194] Registered new tserver with Master: c5750b4a46fa43549d23b89c9ba12114 (127.20.10.129:41187)
I20260807 22:44:26.667265 30737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.129:46493
W20260807 22:44:27.027475 30955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:27.027866 30955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:27.028038 30955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:27.036062 30955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:27.036227 30955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:27.036413 30955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:44:27.047652 30955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:27.049096 30955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:27.051091 30955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:27.065208 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:27.065155 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:27.065589 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:27.066807 30955 server_base.cc:1034] running on GCE node
I20260807 22:44:27.067586 30955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:27.068732 30955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:27.070009 30955 hybrid_clock.cc:648] HybridClock initialized: now 1786142667069956 us; error 43 us; skew 500 ppm
I20260807 22:44:27.070544 30955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:27.073648 30955 webserver.cc:460] Webserver started at http://127.20.10.130:45881/ using document root <none> and password file <none>
I20260807 22:44:27.074596 30955 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:27.074774 30955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:27.075217 30955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:27.082044 30955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "e6ea1643fd354b54991b11633950a78f"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "c83e1f55eedee4cd2a1d50d222c518ce"
server_key_iv: "7d20c478fb62d04c1fb2d0c2beb278f4"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.083053 30955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "e6ea1643fd354b54991b11633950a78f"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "c83e1f55eedee4cd2a1d50d222c518ce"
server_key_iv: "7d20c478fb62d04c1fb2d0c2beb278f4"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.088908 30955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 22:44:27.093034 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.094928 30955 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:44:27.095152 30955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "e6ea1643fd354b54991b11633950a78f"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "c83e1f55eedee4cd2a1d50d222c518ce"
server_key_iv: "7d20c478fb62d04c1fb2d0c2beb278f4"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.095412 30955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:27.118964 30955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:27.119930 30955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:27.120271 30955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:27.121459 30955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:27.123210 30955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:27.123324 30955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.123406 30955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:27.123466 30955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.161945 30955 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44437
I20260807 22:44:27.162045 31093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44437 every 8 connection(s)
I20260807 22:44:27.163421 30955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:27.167395 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 30955
I20260807 22:44:27.167661 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:44:27.168335 20522 external_mini_cluster.cc:1428] Setting key e214357fc4f4cee700377af808ef32e4
I20260807 22:44:27.171900 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--builtin_ntp_servers=127.20.10.148:35001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:44:27.178543 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.166530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:53353 (local address 127.20.10.190:46167)
0807 22:44:27.167139 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.167144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.168682 (+ 1538us) server_negotiation.cc:407] Connection header received
0807 22:44:27.170999 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.171004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.171096 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.171217 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.172758 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.173538 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.175471 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.175919 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.176061 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.177618 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:27.177624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:27.177627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:27.177649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:27.177693 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:27.177705 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:27.177805 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:27.178059 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 22:44:27.178171 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":360,"threads_started":1}
I20260807 22:44:27.179705 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.167241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46167 (local address 127.20.10.130:53353)
0807 22:44:27.168486 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:27.168554 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:27.170703 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:27.171538 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:27.171547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:27.171646 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:27.172557 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:27.172586 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.173710 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.173716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:27.175291 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:27.175306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.176177 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.176184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:27.176440 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.177266 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:27.177269 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:27.177312 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:27.177405 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:27.177842 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:27.177852 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:27.177859 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:44:27.178047 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 22:44:27.178496 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":431,"threads_started":1}
I20260807 22:44:27.181566 31095 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46167
I20260807 22:44:27.181910 31095 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:27.182871 31095 heartbeater.cc:511] Master 127.20.10.190:46167 requested a full tablet report, sending...
I20260807 22:44:27.184666 30737 ts_manager.cc:194] Registered new tserver with Master: e6ea1643fd354b54991b11633950a78f (127.20.10.130:44437)
I20260807 22:44:27.185577 30737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.130:53353
W20260807 22:44:27.550683 31099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:27.551010 31099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:27.551167 31099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:27.559000 31099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:27.559128 31099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:27.559298 31099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:44:27.569308 31099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:35001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:27.570849 31099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:27.573087 31099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:27.587960 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:27.588023 31109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:27.588055 31107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:27.589866 31099 server_base.cc:1034] running on GCE node
I20260807 22:44:27.590566 31099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:27.591813 31099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:27.593101 31099 hybrid_clock.cc:648] HybridClock initialized: now 1786142667593031 us; error 87 us; skew 500 ppm
I20260807 22:44:27.593721 31099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:27.596871 31099 webserver.cc:460] Webserver started at http://127.20.10.131:37327/ using document root <none> and password file <none>
I20260807 22:44:27.597719 31099 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:27.597843 31099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:27.598219 31099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:27.601609 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "51a16dc7f48143818e4e01f5a61d98e5"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "1200594e0c177419898ac92650bb1512"
server_key_iv: "46e88602c16052fdbc39ded8121c6808"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.602566 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "51a16dc7f48143818e4e01f5a61d98e5"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "1200594e0c177419898ac92650bb1512"
server_key_iv: "46e88602c16052fdbc39ded8121c6808"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.608979 31099 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 22:44:27.613397 31116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.615193 31099 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:44:27.615429 31099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "51a16dc7f48143818e4e01f5a61d98e5"
format_stamp: "Formatted at 2026-08-07 22:44:27 on dist-test-slave-csg5"
server_key: "1200594e0c177419898ac92650bb1512"
server_key_iv: "46e88602c16052fdbc39ded8121c6808"
server_key_version: "encryptionkey@0"
I20260807 22:44:27.615696 31099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:27.635157 31099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:27.636214 31099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:27.636579 31099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:27.637739 31099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:27.639647 31099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:27.639792 31099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.639976 31099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:27.640076 31099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:27.672060 30951 heartbeater.cc:503] Master 127.20.10.190:46167 was elected leader, sending a full tablet report...
I20260807 22:44:27.679662 31099 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:45459
I20260807 22:44:27.679729 31234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:45459 every 8 connection(s)
I20260807 22:44:27.681447 31099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:27.688377 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31099
I20260807 22:44:27.688604 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:44:27.689181 20522 external_mini_cluster.cc:1428] Setting key 382a7364263d5e33a3a0e30c7a913f38
I20260807 22:44:27.696353 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.684587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:35223 (local address 127.20.10.190:46167)
0807 22:44:27.685168 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.685178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.686516 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 22:44:27.688758 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.688763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.688855 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.688968 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.690405 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.691513 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.693214 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.693604 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.693749 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.695364 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:27.695369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:27.695373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:27.695393 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:27.695443 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:27.695456 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:27.695553 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:27.695798 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 22:44:27.695918 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":304,"threads_started":1}
I20260807 22:44:27.697531 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.685242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46167 (local address 127.20.10.131:35223)
0807 22:44:27.686324 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:27.686376 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:27.688350 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:27.689283 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:27.689297 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:27.689437 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:27.690208 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:27.690229 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.691695 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.691700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:27.693027 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:27.693042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.693861 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.693871 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:27.694134 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.694973 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:27.694976 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:27.695023 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:27.695118 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:27.695555 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:27.695571 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:27.695581 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:44:27.695786 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 22:44:27.696259 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":369,"threads_started":1}
I20260807 22:44:27.699586 31235 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46167
I20260807 22:44:27.699954 31235 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:27.700990 31235 heartbeater.cc:511] Master 127.20.10.190:46167 requested a full tablet report, sending...
I20260807 22:44:27.702953 30737 ts_manager.cc:194] Registered new tserver with Master: 51a16dc7f48143818e4e01f5a61d98e5 (127.20.10.131:45459)
I20260807 22:44:27.703991 30737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.131:35223
I20260807 22:44:27.706058 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:44:27.713882 31237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:27.711712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.20.10.190:46167)
0807 22:44:27.711957 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.711966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.712688 (+ 722us) server_negotiation.cc:407] Connection header received
0807 22:44:27.712880 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.712885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.712963 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.713065 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.713631 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46264: BlockingRecv error: recv got EOF from 127.0.0.1:46264 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:44:27.714941 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30804
I20260807 22:44:27.725598 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.716060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.20.10.129:41187)
0807 22:44:27.716854 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.716862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.716892 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:44:27.716960 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.716966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.717236 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.717380 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.717957 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.719261 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.721353 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.722020 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.722179 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.723775 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:27.723785 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:27.723801 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:27.724545 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:27.724606 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:27.724634 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:27.724749 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:27.724973 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 22:44:27.725228 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":394,"threads_started":1}
I20260807 22:44:27.727159 30945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:27.814567 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30804
I20260807 22:44:27.837160 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30955
I20260807 22:44:27.846275 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.838100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.20.10.130:44437)
0807 22:44:27.838748 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.838756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.838822 (+ 66us) server_negotiation.cc:407] Connection header received
0807 22:44:27.838888 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.838894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.839170 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.839316 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.839802 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.840853 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.842234 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.842887 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.843014 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.844484 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:27.844492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:27.844507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:27.845250 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:27.845313 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:27.845335 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:27.845442 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:27.845654 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 22:44:27.845862 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":232,"threads_started":1}
I20260807 22:44:27.847782 31088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:27.933992 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30955
I20260807 22:44:27.958782 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31099
I20260807 22:44:27.968111 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:27.960012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.20.10.131:45459)
0807 22:44:27.960727 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:27.960734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:27.960802 (+ 68us) server_negotiation.cc:407] Connection header received
0807 22:44:27.960876 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:27.960883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:27.961129 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:27.961331 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:27.961843 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.963018 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.964411 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:27.965033 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:27.965194 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:27.966471 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:27.966480 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:27.966495 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:27.967186 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:27.967245 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:27.967269 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:27.967404 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:27.967560 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:44:27.967726 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":382,"threads_started":1}
I20260807 22:44:27.969467 31230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:28.066232 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31099
I20260807 22:44:28.087873 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 30703
I20260807 22:44:28.089221 30772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:28.199780 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 30703
2026-08-07T22:44:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2797 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T22:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:28Z Disabled control of system clock
I20260807 22:44:28.299201 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39615
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:44:28.694952 31260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:28.695319 31260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:28.695420 31260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:28.703579 31260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:28.703729 31260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:28.703784 31260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:28.703841 31260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:28.703889 31260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:28.713816 31260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:39615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:28.715266 31260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:28.717226 31260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:28.731973 31265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:28.731973 31266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:28.731904 31269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:28.733171 31260 server_base.cc:1034] running on GCE node
I20260807 22:44:28.734210 31260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:28.736114 31260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:28.737418 31260 hybrid_clock.cc:648] HybridClock initialized: now 1786142668737378 us; error 62 us; skew 500 ppm
I20260807 22:44:28.737849 31260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:28.744663 31260 webserver.cc:460] Webserver started at http://127.20.10.190:34581/ using document root <none> and password file <none>
I20260807 22:44:28.745674 31260 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:28.745857 31260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:28.746248 31260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:28.749832 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6e4a9d1d0d6243cd866783bf2bcb8406"
format_stamp: "Formatted at 2026-08-07 22:44:28 on dist-test-slave-csg5"
server_key: "db5d710377c868c51313535db2b263dd"
server_key_iv: "0ca24214d2bb6633b32788e67e5ad95e"
server_key_version: "encryptionkey@0"
I20260807 22:44:28.750906 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6e4a9d1d0d6243cd866783bf2bcb8406"
format_stamp: "Formatted at 2026-08-07 22:44:28 on dist-test-slave-csg5"
server_key: "db5d710377c868c51313535db2b263dd"
server_key_iv: "0ca24214d2bb6633b32788e67e5ad95e"
server_key_version: "encryptionkey@0"
I20260807 22:44:28.757097 31260 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:44:28.761801 31275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:28.763569 31260 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:44:28.763829 31260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6e4a9d1d0d6243cd866783bf2bcb8406"
format_stamp: "Formatted at 2026-08-07 22:44:28 on dist-test-slave-csg5"
server_key: "db5d710377c868c51313535db2b263dd"
server_key_iv: "0ca24214d2bb6633b32788e67e5ad95e"
server_key_version: "encryptionkey@0"
I20260807 22:44:28.764094 31260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:28.770550 31260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:28.771513 31260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:28.771874 31260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:28.794106 31260 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:39615
I20260807 22:44:28.794153 31331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:39615 every 8 connection(s)
I20260807 22:44:28.795634 31260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:28.800370 31332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:28.803073 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31260
I20260807 22:44:28.803325 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:44:28.804234 20522 external_mini_cluster.cc:1428] Setting key f1775b295de242ef39397977989849f7
I20260807 22:44:28.814208 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:28.806920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.20.10.190:39615)
0807 22:44:28.808157 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:28.808171 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:28.808223 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:44:28.810042 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:28.810067 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:28.810527 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:28.810993 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:28.812333 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:28.812340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:28.812353 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:28.812370 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:28.812440 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:28.812460 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:28.812727 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:28.812885 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:44:28.813174 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":433,"threads_started":1}
I20260807 22:44:28.815616 31332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406: Bootstrap starting.
I20260807 22:44:28.819554 31332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:28.820741 31332 log.cc:824] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:28.824411 31332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406: No bootstrap required, opened a new log
I20260807 22:44:28.833392 31332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } }
I20260807 22:44:28.833841 31332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:28.833942 31332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4a9d1d0d6243cd866783bf2bcb8406, State: Initialized, Role: FOLLOWER
I20260807 22:44:28.834632 31332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } }
I20260807 22:44:28.835039 31332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:28.835250 31332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:44:28.835472 31332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:28.838084 31332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } }
I20260807 22:44:28.838614 31332 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4a9d1d0d6243cd866783bf2bcb8406; no voters:
I20260807 22:44:28.839430 31332 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:44:28.839617 31337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:28.841192 31337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [term 1 LEADER]: Becoming Leader. State: Replica: 6e4a9d1d0d6243cd866783bf2bcb8406, State: Running, Role: LEADER
I20260807 22:44:28.841845 31337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } }
I20260807 22:44:28.842146 31332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:28.844920 31339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4a9d1d0d6243cd866783bf2bcb8406. Latest consensus state: current_term: 1 leader_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } } }
I20260807 22:44:28.845464 31339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:28.844926 31338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4a9d1d0d6243cd866783bf2bcb8406" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39615 } } }
I20260807 22:44:28.845885 31338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:28.847380 31343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:28.852943 31343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:28.862998 31343 catalog_manager.cc:1347] Generated new cluster ID: 512fcdc11fa340d2b7fc0310f333df1e
I20260807 22:44:28.863149 31343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:28.875627 31343 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:44:28.876223 31343 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:28.892494 31343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e4a9d1d0d6243cd866783bf2bcb8406: Generated new TSK 0
I20260807 22:44:28.893826 31343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:44:28.905814 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--builtin_ntp_servers=127.20.10.148:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:44:29.298491 31356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:29.298959 31356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:29.299137 31356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:29.306996 31356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:29.307118 31356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:29.307286 31356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:44:29.317220 31356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:29.318648 31356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:29.320668 31356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:29.336894 31367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:29.336936 31364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:29.336941 31365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:29.338534 31356 server_base.cc:1034] running on GCE node
I20260807 22:44:29.339341 31356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:29.340507 31356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:29.341780 31356 hybrid_clock.cc:648] HybridClock initialized: now 1786142669341723 us; error 43 us; skew 500 ppm
I20260807 22:44:29.342221 31356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:29.345305 31356 webserver.cc:460] Webserver started at http://127.20.10.129:46421/ using document root <none> and password file <none>
I20260807 22:44:29.346170 31356 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:29.346326 31356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:29.346781 31356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:29.351537 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "2af53847375640dabe27c7cb847d1a51"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "47835b93c20d2e838112fdb4e7678cb2"
server_key_iv: "fd44c536c78d52ad8b5c13895b351a77"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.352981 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "2af53847375640dabe27c7cb847d1a51"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "47835b93c20d2e838112fdb4e7678cb2"
server_key_iv: "fd44c536c78d52ad8b5c13895b351a77"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.361781 31356 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 22:44:29.367547 31373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.369733 31356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 22:44:29.369971 31356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "2af53847375640dabe27c7cb847d1a51"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "47835b93c20d2e838112fdb4e7678cb2"
server_key_iv: "fd44c536c78d52ad8b5c13895b351a77"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.370301 31356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:29.378607 31356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:29.379671 31356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:29.379980 31356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:29.381234 31356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:29.383154 31356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:29.383291 31356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.383425 31356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:29.383519 31356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.422685 31356 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41761
I20260807 22:44:29.422741 31494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41761 every 8 connection(s)
I20260807 22:44:29.424127 31356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:29.432482 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31356
I20260807 22:44:29.432672 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:44:29.433488 20522 external_mini_cluster.cc:1428] Setting key 6da971b9e82704a9ab38d79ecd4da698
I20260807 22:44:29.435058 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:29.428661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:51525 (local address 127.20.10.190:39615)
0807 22:44:29.429246 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:29.429254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:29.430004 (+ 750us) server_negotiation.cc:407] Connection header received
0807 22:44:29.432043 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:29.432052 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:29.432154 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:29.432314 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:29.434132 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:29.434143 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:29.434147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:29.434169 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:29.434220 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:29.434233 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:29.434374 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:29.434586 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 22:44:29.434680 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":294,"threads_started":1}
I20260807 22:44:29.436774 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:29.428030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39615 (local address 127.20.10.129:51525)
0807 22:44:29.429815 (+ 1785us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:29.429864 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:29.431679 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:29.432784 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:29.432803 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:29.432919 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:29.433766 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:29.433769 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:29.433822 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:29.433923 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:29.434351 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:29.434366 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:29.434375 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 22:44:29.434773 (+ 398us) client_negotiation.cc:240] Negotiation successful
0807 22:44:29.435150 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"thread_start_us":880,"threads_started":1}
I20260807 22:44:29.437305 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--builtin_ntp_servers=127.20.10.148:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:44:29.438854 31495 heartbeater.cc:347] Connected to a master server at 127.20.10.190:39615
I20260807 22:44:29.439276 31495 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:29.440121 31495 heartbeater.cc:511] Master 127.20.10.190:39615 requested a full tablet report, sending...
I20260807 22:44:29.442816 31294 ts_manager.cc:194] Registered new tserver with Master: 2af53847375640dabe27c7cb847d1a51 (127.20.10.129:41761)
W20260807 22:44:29.835465 31500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:29.835781 31500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:29.835935 31500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:29.843632 31500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:29.843760 31500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:29.843991 31500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:44:29.854008 31500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:29.855654 31500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:29.857658 31500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:29.877344 31508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:29.878388 31507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:29.879845 31510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:29.879845 31500 server_base.cc:1034] running on GCE node
I20260807 22:44:29.880892 31500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:29.882068 31500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:29.883333 31500 hybrid_clock.cc:648] HybridClock initialized: now 1786142669883245 us; error 43 us; skew 500 ppm
I20260807 22:44:29.883714 31500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:29.886566 31500 webserver.cc:460] Webserver started at http://127.20.10.130:38545/ using document root <none> and password file <none>
I20260807 22:44:29.887514 31500 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:29.887630 31500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:29.888017 31500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:29.891369 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "43fa022c61724b6394c9ff4b6ae0db37"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "6bff20a6c6da5318e03efc4bd85a7790"
server_key_iv: "83e5458b5bfb48c23b7daa651fb56f22"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.892277 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "43fa022c61724b6394c9ff4b6ae0db37"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "6bff20a6c6da5318e03efc4bd85a7790"
server_key_iv: "83e5458b5bfb48c23b7daa651fb56f22"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.898521 31500 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:44:29.903153 31516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.905064 31500 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:44:29.905339 31500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "43fa022c61724b6394c9ff4b6ae0db37"
format_stamp: "Formatted at 2026-08-07 22:44:29 on dist-test-slave-csg5"
server_key: "6bff20a6c6da5318e03efc4bd85a7790"
server_key_iv: "83e5458b5bfb48c23b7daa651fb56f22"
server_key_version: "encryptionkey@0"
I20260807 22:44:29.905627 31500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:29.912907 31500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:29.913936 31500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:29.914283 31500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:29.915571 31500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:29.917312 31500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:29.917449 31500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.917598 31500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:29.917697 31500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:29.958635 31500 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44259
I20260807 22:44:29.958724 31638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44259 every 8 connection(s)
I20260807 22:44:29.960212 31500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:29.962488 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31500
I20260807 22:44:29.962630 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:44:29.963192 20522 external_mini_cluster.cc:1428] Setting key 41d50a8cecf07932ca14d661f2705dba
I20260807 22:44:29.967180 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--builtin_ntp_servers=127.20.10.148:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:44:29.970721 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:29.963395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:59467 (local address 127.20.10.190:39615)
0807 22:44:29.963969 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:29.963974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:29.965511 (+ 1537us) server_negotiation.cc:407] Connection header received
0807 22:44:29.967718 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:29.967727 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:29.967891 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:29.968034 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:29.969658 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:29.969668 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:29.969676 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:29.969704 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:29.969781 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:29.969802 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:29.969968 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:29.970122 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:44:29.970205 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":279,"threads_started":1}
I20260807 22:44:29.971570 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:29.964175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39615 (local address 127.20.10.130:59467)
0807 22:44:29.965339 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:29.965389 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:29.967320 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:29.968325 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:29.968334 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:29.968459 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:29.969321 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:29.969323 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:29.969365 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:29.969462 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:29.969915 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:29.969923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:29.969929 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:44:29.970162 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 22:44:29.970385 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":371,"threads_started":1}
I20260807 22:44:29.973770 31639 heartbeater.cc:347] Connected to a master server at 127.20.10.190:39615
I20260807 22:44:29.974282 31639 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:29.975419 31639 heartbeater.cc:511] Master 127.20.10.190:39615 requested a full tablet report, sending...
I20260807 22:44:29.977720 31294 ts_manager.cc:194] Registered new tserver with Master: 43fa022c61724b6394c9ff4b6ae0db37 (127.20.10.130:44259)
W20260807 22:44:30.364755 31643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:30.365154 31643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:30.365335 31643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:30.373834 31643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:30.373991 31643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:30.374168 31643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:44:30.384116 31643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:30.385751 31643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:30.387745 31643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:30.403076 31649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:30.403091 31648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:30.404011 31651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:30.404348 31643 server_base.cc:1034] running on GCE node
I20260807 22:44:30.405129 31643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:30.406240 31643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:30.407517 31643 hybrid_clock.cc:648] HybridClock initialized: now 1786142670407462 us; error 50 us; skew 500 ppm
I20260807 22:44:30.407943 31643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:44:30.411854 31643 webserver.cc:460] Webserver started at http://127.20.10.131:42231/ using document root <none> and password file <none>
I20260807 22:44:30.412710 31643 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:30.412856 31643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:30.413268 31643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:30.416746 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "21d12052399c43b2a302a575698ecf40"
format_stamp: "Formatted at 2026-08-07 22:44:30 on dist-test-slave-csg5"
server_key: "b8a6a233bf0c0b865d2c3695b5cf6761"
server_key_iv: "f1fcc4459457111a311e97622bbc6234"
server_key_version: "encryptionkey@0"
I20260807 22:44:30.417680 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "21d12052399c43b2a302a575698ecf40"
format_stamp: "Formatted at 2026-08-07 22:44:30 on dist-test-slave-csg5"
server_key: "b8a6a233bf0c0b865d2c3695b5cf6761"
server_key_iv: "f1fcc4459457111a311e97622bbc6234"
server_key_version: "encryptionkey@0"
I20260807 22:44:30.424366 31643 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 22:44:30.428797 31658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:30.430918 31643 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:44:30.431123 31643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "21d12052399c43b2a302a575698ecf40"
format_stamp: "Formatted at 2026-08-07 22:44:30 on dist-test-slave-csg5"
server_key: "b8a6a233bf0c0b865d2c3695b5cf6761"
server_key_iv: "f1fcc4459457111a311e97622bbc6234"
server_key_version: "encryptionkey@0"
I20260807 22:44:30.431550 31643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:30.439383 31643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:30.440332 31643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:30.440609 31643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:30.441668 31643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:30.443640 31643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:30.443759 31643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:30.443894 31643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:30.443966 31643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:30.446527 31495 heartbeater.cc:503] Master 127.20.10.190:39615 was elected leader, sending a full tablet report...
I20260807 22:44:30.483772 31643 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:41567
I20260807 22:44:30.483880 31781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:41567 every 8 connection(s)
I20260807 22:44:30.485373 31643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:30.494853 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31643
I20260807 22:44:30.495062 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:44:30.495769 20522 external_mini_cluster.cc:1428] Setting key 928c8819952621ac77061cbf9fe54d4b
I20260807 22:44:30.496330 31784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:30.488912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:40555 (local address 127.20.10.190:39615)
0807 22:44:30.489459 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:30.489467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:30.490999 (+ 1532us) server_negotiation.cc:407] Connection header received
0807 22:44:30.493006 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:30.493011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:30.493073 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:30.493188 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:30.495226 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:30.495243 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:30.495251 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:30.495294 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:30.495360 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:30.495384 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:30.495550 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:30.495823 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 22:44:30.495910 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":244,"threads_started":1}
I20260807 22:44:30.497607 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:30.489477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39615 (local address 127.20.10.131:40555)
0807 22:44:30.490788 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:30.490855 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:30.492703 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:30.493602 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:30.493613 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:30.493719 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:30.494600 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:44:30.494603 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:30.494649 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:44:30.494856 (+ 207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:30.495558 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:30.495568 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:30.495575 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:44:30.495884 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 22:44:30.496246 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":342,"threads_started":1}
I20260807 22:44:30.499327 31782 heartbeater.cc:347] Connected to a master server at 127.20.10.190:39615
I20260807 22:44:30.499647 31782 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:30.500280 31782 heartbeater.cc:511] Master 127.20.10.190:39615 requested a full tablet report, sending...
I20260807 22:44:30.501999 31294 ts_manager.cc:194] Registered new tserver with Master: 21d12052399c43b2a302a575698ecf40 (127.20.10.131:41567)
I20260807 22:44:30.512715 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:44:30.520002 31784 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:44:30.517910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.20.10.190:39615)
0807 22:44:30.518111 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:30.518116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:30.518907 (+ 791us) server_negotiation.cc:407] Connection header received
0807 22:44:30.519049 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:30.519053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:30.519112 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:30.519213 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:30.519684 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33560: BlockingRecv error: recv got EOF from 127.0.0.1:33560 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:44:30.521247 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31356
I20260807 22:44:30.525854 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:30.522548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.20.10.129:41761)
0807 22:44:30.523285 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:30.523293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:30.523323 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:44:30.523388 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:30.523395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:30.523635 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:30.523787 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:30.524988 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:30.524996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:30.525009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:30.525026 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:30.525091 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:30.525109 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:30.525366 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:30.525424 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:44:30.525522 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":386,"threads_started":1}
I20260807 22:44:30.527241 31490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:30.623915 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31356
I20260807 22:44:30.647960 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31500
I20260807 22:44:30.652231 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:30.649103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.20.10.130:44259)
0807 22:44:30.649767 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:30.649775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:30.649804 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:44:30.649874 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:30.649880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:30.650112 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:30.650261 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:30.651465 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:30.651473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:30.651486 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:30.651500 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:30.651553 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:30.651569 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:30.651707 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:30.651877 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:44:30.651979 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":343,"threads_started":1}
I20260807 22:44:30.653863 31634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:30.740502 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31500
I20260807 22:44:30.764745 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31643
I20260807 22:44:30.769336 31797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:30.765862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.20.10.131:41567)
0807 22:44:30.766547 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:30.766554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:30.766587 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:44:30.766703 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:30.766717 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:30.766992 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:30.767178 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:30.768492 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:30.768499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:30.768511 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:44:30.768529 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:30.768570 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:44:30.768584 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:30.768785 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:30.768867 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 22:44:30.768986 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":339,"threads_started":1}
I20260807 22:44:30.771155 31777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:30.858753 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31643
I20260807 22:44:30.882025 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31260
I20260807 22:44:30.883620 31327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:30.995666 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31260
2026-08-07T22:44:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2799 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:44:31 dist-test-slave-csg5 krb5kdc[31802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:44:31 dist-test-slave-csg5 krb5kdc[31802](info): set up 2 sockets
Aug 07 22:44:31 dist-test-slave-csg5 krb5kdc[31802](info): commencing operation
krb5kdc: starting...
W20260807 22:44:32.300164 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:44:32 dist-test-slave-csg5 krb5kdc[31802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:44:32.647861 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36673
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:36673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:33.019405 31815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:33.019804 31815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:33.019973 31815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:33.027720 31815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:33.027908 31815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:33.028024 31815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:33.028129 31815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:33.028224 31815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:33.039157 31815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:36673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:33.040778 31815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:33.042258 31815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:33.056088 31820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:33.056026 31821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:33.056141 31824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:33.057559 31815 server_base.cc:1034] running on GCE node
I20260807 22:44:33.058494 31815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:33.060207 31815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:33.061465 31815 hybrid_clock.cc:648] HybridClock initialized: now 1786142673061417 us; error 45 us; skew 500 ppm
I20260807 22:44:33.061841 31815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:33 dist-test-slave-csg5 krb5kdc[31802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:33.065580 31815 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:44:33.067369 31815 webserver.cc:460] Webserver started at http://127.20.10.190:35499/ using document root <none> and password file <none>
I20260807 22:44:33.068109 31815 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:33.068267 31815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:33.068595 31815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:33.071913 31815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "166297ae108a483a876fe886a34db5c3"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "4474e5bf4949b21eb464006f6d70104f"
server_key_iv: "0d7b7448477b4cb9f93695bd9dafe456"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.072788 31815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "166297ae108a483a876fe886a34db5c3"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "4474e5bf4949b21eb464006f6d70104f"
server_key_iv: "0d7b7448477b4cb9f93695bd9dafe456"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.078917 31815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 22:44:33.083240 31832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:33.085000 31815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:44:33.085268 31815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "166297ae108a483a876fe886a34db5c3"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "4474e5bf4949b21eb464006f6d70104f"
server_key_iv: "0d7b7448477b4cb9f93695bd9dafe456"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.085558 31815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:33.108878 31815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:33.114589 31815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:33.114945 31815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:33.134862 31815 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:36673
I20260807 22:44:33.134920 31890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:36673 every 8 connection(s)
I20260807 22:44:33.136425 31815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:33.140769 31891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:33.141498 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31815
I20260807 22:44:33.141767 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:44:33.142441 20522 external_mini_cluster.cc:1428] Setting key 6e5ecf95636398349e4e2a45475a3a65
I20260807 22:44:33.156781 31891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: Bootstrap starting.
Aug 07 22:44:33 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:33.160586 31891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:33.162137 31891 log.cc:824] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:33.166422 31891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: No bootstrap required, opened a new log
I20260807 22:44:33.171530 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:33.144706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.20.10.190:36673)
0807 22:44:33.145628 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:33.145647 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:33.145703 (+ 56us) server_negotiation.cc:407] Connection header received
0807 22:44:33.148058 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:33.148093 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:33.148742 (+ 649us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:33.149303 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:33.151236 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.153412 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.155092 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.156227 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.156448 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:33.161087 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:33.161103 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:33.161126 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:33.161198 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:33.165714 (+ 4516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:33.166890 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:33.166900 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:33.166909 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:33.167172 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:33.167659 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:33.167664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:33.167667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:33.168531 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:33.168883 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:33.169396 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 22:44:33.169928 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":382,"threads_started":1}
I20260807 22:44:33.176999 31891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } }
I20260807 22:44:33.177441 31891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:33.177559 31891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166297ae108a483a876fe886a34db5c3, State: Initialized, Role: FOLLOWER
I20260807 22:44:33.178224 31891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } }
I20260807 22:44:33.178447 31891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:33.178606 31891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:44:33.178875 31891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:33.181200 31891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } }
I20260807 22:44:33.181782 31891 leader_election.cc:302] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 166297ae108a483a876fe886a34db5c3; no voters:
I20260807 22:44:33.182515 31891 leader_election.cc:288] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:44:33.182735 31896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:33.184393 31896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [term 1 LEADER]: Becoming Leader. State: Replica: 166297ae108a483a876fe886a34db5c3, State: Running, Role: LEADER
I20260807 22:44:33.185436 31896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } }
I20260807 22:44:33.185640 31891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:33.189239 31898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166297ae108a483a876fe886a34db5c3. Latest consensus state: current_term: 1 leader_uuid: "166297ae108a483a876fe886a34db5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } } }
I20260807 22:44:33.189980 31898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:33.189226 31897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "166297ae108a483a876fe886a34db5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166297ae108a483a876fe886a34db5c3" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36673 } } }
I20260807 22:44:33.191030 31897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:33.193002 31903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:33.201156 31903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:33.212288 31903 catalog_manager.cc:1347] Generated new cluster ID: ba4047daa5124466860094943aaa822d
I20260807 22:44:33.212451 31903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:33.230335 31903 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:44:33.231977 31903 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:33.239357 31903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: Generated new TSK 0
I20260807 22:44:33.240561 31903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:44:33.328549 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:33.744221 31920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:33.744544 31920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:33.744702 31920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:33.752905 31920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:33.753047 31920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:33.753247 31920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:44:33.764221 31920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:33.765856 31920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:33.767354 31920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:33.782789 31926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:33.782884 31929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:33.782801 31927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:33.784549 31920 server_base.cc:1034] running on GCE node
I20260807 22:44:33.785454 31920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:33.786612 31920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:33.787971 31920 hybrid_clock.cc:648] HybridClock initialized: now 1786142673787901 us; error 102 us; skew 500 ppm
I20260807 22:44:33.788432 31920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:33 dist-test-slave-csg5 krb5kdc[31802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:33.797802 31920 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:44:33.799777 31920 webserver.cc:460] Webserver started at http://127.20.10.129:45989/ using document root <none> and password file <none>
I20260807 22:44:33.800526 31920 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:33.800643 31920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:33.800926 31920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:33.804147 31920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "9711c65ebe6046d0b2ced066ba55277b"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "976d60c8e337cdd0323734d75213459f"
server_key_iv: "8352de61c9960d52c703ba3694d68e4c"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.804930 31920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "9711c65ebe6046d0b2ced066ba55277b"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "976d60c8e337cdd0323734d75213459f"
server_key_iv: "8352de61c9960d52c703ba3694d68e4c"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.810559 31920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 22:44:33.814642 31936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:33.816615 31920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 22:44:33.816812 31920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "9711c65ebe6046d0b2ced066ba55277b"
format_stamp: "Formatted at 2026-08-07 22:44:33 on dist-test-slave-csg5"
server_key: "976d60c8e337cdd0323734d75213459f"
server_key_iv: "8352de61c9960d52c703ba3694d68e4c"
server_key_version: "encryptionkey@0"
I20260807 22:44:33.817094 31920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:33.838150 31920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:33.843392 31920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:33.843806 31920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:33.844988 31920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:33.846644 31920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:33.846781 31920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 22:44:33.846957 31920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:33.847029 31920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:33.885804 31920 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:34269
I20260807 22:44:33.885870 32061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:34269 every 8 connection(s)
I20260807 22:44:33.887516 31920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:33.895795 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 31920
I20260807 22:44:33.896039 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:44:33.896888 20522 external_mini_cluster.cc:1428] Setting key bd474ae2c91de7fa181d1efd78396fb5
Aug 07 22:44:33 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:33.914757 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:33.890557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:59717 (local address 127.20.10.190:36673)
0807 22:44:33.891148 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:33.891153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:33.892063 (+ 910us) server_negotiation.cc:407] Connection header received
0807 22:44:33.894353 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:33.894359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:33.894490 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:33.894613 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:33.897095 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.898135 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.901221 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.901653 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.901786 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:33.907650 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:33.907656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:33.907660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:33.907712 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:33.910818 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:33.911845 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:33.911851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:33.911854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:33.912007 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:33.912555 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:33.912559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:33.912562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:33.912889 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:33.913083 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:33.913980 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 22:44:33.914292 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":331,"threads_started":1}
I20260807 22:44:33.915881 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:33.891109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36673 (local address 127.20.10.129:59717)
0807 22:44:33.891885 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:33.891937 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:33.894001 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:33.894915 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:33.894926 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:33.895788 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:33.896775 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:33.896807 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.898439 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.898444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:33.900955 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:33.900983 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:33.901982 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:33.901994 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:33.902431 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:33.903570 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:33.903619 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:33.907287 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:33.911036 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:33.911048 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:33.911076 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:33.911667 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:33.912148 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:33.912155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:33.912160 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:33.912407 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:33.913052 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:33.913061 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:33.913510 (+ 449us) client_negotiation.cc:769] Sending connection context
0807 22:44:33.913831 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 22:44:33.914494 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":290,"threads_started":1}
I20260807 22:44:33.917995 32062 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36673
I20260807 22:44:33.918450 32062 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:33.919610 32062 heartbeater.cc:511] Master 127.20.10.190:36673 requested a full tablet report, sending...
I20260807 22:44:33.922900 31855 ts_manager.cc:194] Registered new tserver with Master: 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269)
I20260807 22:44:33.925612 31855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:59717
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:44:33.979990 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:34.369691 32069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:34.370014 32069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:34.370165 32069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:34.377938 32069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:34.378063 32069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:34.378300 32069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:44:34.389142 32069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:34.390859 32069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:34.392275 32069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:34.407261 32074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:34.407430 32077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:34.407292 32075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:34.408687 32069 server_base.cc:1034] running on GCE node
I20260807 22:44:34.409524 32069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:34.410763 32069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:34.412021 32069 hybrid_clock.cc:648] HybridClock initialized: now 1786142674411965 us; error 48 us; skew 500 ppm
I20260807 22:44:34.412389 32069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:34 dist-test-slave-csg5 krb5kdc[31802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:34.416155 32069 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:44:34.417870 32069 webserver.cc:460] Webserver started at http://127.20.10.130:39869/ using document root <none> and password file <none>
I20260807 22:44:34.418691 32069 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:34.418859 32069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:34.419212 32069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:34.422616 32069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "6e64e2221d8e44ec8d00b1d86796811c"
format_stamp: "Formatted at 2026-08-07 22:44:34 on dist-test-slave-csg5"
server_key: "4664377d210082c5e723364c27c0ad2c"
server_key_iv: "cb6f5305d4016bc74ae32e34fa5047e4"
server_key_version: "encryptionkey@0"
I20260807 22:44:34.423599 32069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "6e64e2221d8e44ec8d00b1d86796811c"
format_stamp: "Formatted at 2026-08-07 22:44:34 on dist-test-slave-csg5"
server_key: "4664377d210082c5e723364c27c0ad2c"
server_key_iv: "cb6f5305d4016bc74ae32e34fa5047e4"
server_key_version: "encryptionkey@0"
I20260807 22:44:34.429762 32069 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:44:34.434297 32086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:34.436190 32069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:44:34.436434 32069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "6e64e2221d8e44ec8d00b1d86796811c"
format_stamp: "Formatted at 2026-08-07 22:44:34 on dist-test-slave-csg5"
server_key: "4664377d210082c5e723364c27c0ad2c"
server_key_iv: "cb6f5305d4016bc74ae32e34fa5047e4"
server_key_version: "encryptionkey@0"
I20260807 22:44:34.436687 32069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:34.459549 32069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:34.465284 32069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:34.465689 32069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:34.467000 32069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:34.468683 32069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:34.468801 32069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:34.468976 32069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:34.469058 32069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:34.509371 32069 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:41827
I20260807 22:44:34.509467 32214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:41827 every 8 connection(s)
I20260807 22:44:34.510936 32069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:34.516654 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 32069
I20260807 22:44:34.516875 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:44:34.517460 20522 external_mini_cluster.cc:1428] Setting key 6c4e1d570b2aa8efcd091c660dea8706
Aug 07 22:44:34 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:34.538913 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:34.514181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:42373 (local address 127.20.10.190:36673)
0807 22:44:34.514745 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:34.514751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:34.516000 (+ 1249us) server_negotiation.cc:407] Connection header received
0807 22:44:34.518788 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:34.518795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:34.518901 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:34.519029 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:34.521666 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:34.522545 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:34.525537 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:34.526090 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:34.526258 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:34.531698 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:34.531705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:34.531708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:34.531752 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:34.535205 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:34.536197 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:34.536203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:34.536211 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:34.536375 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:34.536880 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:34.536884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:34.536887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:34.537198 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:34.537368 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:34.538158 (+ 790us) server_negotiation.cc:299] Negotiation successful
0807 22:44:34.538423 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":243,"threads_started":1}
I20260807 22:44:34.540035 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:34.514895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36673 (local address 127.20.10.130:42373)
0807 22:44:34.515812 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:34.515854 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:34.518398 (+ 2544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:34.519420 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:34.519430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:34.520362 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:34.521427 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:34.521463 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:34.522755 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:34.522764 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:34.525288 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:34.525315 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:34.526427 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:34.526439 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:34.526876 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:34.527830 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:34.527879 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:34.531353 (+ 3474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:34.535431 (+ 4078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:34.535440 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:34.535457 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:34.536025 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:34.536503 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:34.536509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:34.536512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:34.536734 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:34.537349 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:34.537358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:34.537762 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 22:44:34.538054 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 22:44:34.538747 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":432,"threads_started":1}
I20260807 22:44:34.542032 32215 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36673
I20260807 22:44:34.542450 32215 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:34.543385 32215 heartbeater.cc:511] Master 127.20.10.190:36673 requested a full tablet report, sending...
I20260807 22:44:34.545138 31855 ts_manager.cc:194] Registered new tserver with Master: 6e64e2221d8e44ec8d00b1d86796811c (127.20.10.130:41827)
I20260807 22:44:34.546072 31855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:42373
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:44:34.600523 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:44:34.931071 32062 heartbeater.cc:503] Master 127.20.10.190:36673 was elected leader, sending a full tablet report...
W20260807 22:44:34.985937 32221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:34.986296 32221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:34.986465 32221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:34.994463 32221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:34.994580 32221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:34.994792 32221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:44:35.005230 32221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:44:35.006593 32221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:35.008021 32221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:35.023229 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:35.023104 32229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:35.023258 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:35.023942 32221 server_base.cc:1034] running on GCE node
I20260807 22:44:35.024696 32221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:35.025799 32221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:35.027051 32221 hybrid_clock.cc:648] HybridClock initialized: now 1786142675026985 us; error 57 us; skew 500 ppm
I20260807 22:44:35.027442 32221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:44:35 dist-test-slave-csg5 krb5kdc[31802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:35.031360 32221 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:44:35.033093 32221 webserver.cc:460] Webserver started at http://127.20.10.131:38855/ using document root <none> and password file <none>
I20260807 22:44:35.033906 32221 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:35.034071 32221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:35.034399 32221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:44:35.037858 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "9b6983bcac0f46d4927da6fdee65791e"
format_stamp: "Formatted at 2026-08-07 22:44:35 on dist-test-slave-csg5"
server_key: "4079d22022999ae1f28ef134aac18c2d"
server_key_iv: "fb1633691aa861077263245d328f5963"
server_key_version: "encryptionkey@0"
I20260807 22:44:35.038771 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "9b6983bcac0f46d4927da6fdee65791e"
format_stamp: "Formatted at 2026-08-07 22:44:35 on dist-test-slave-csg5"
server_key: "4079d22022999ae1f28ef134aac18c2d"
server_key_iv: "fb1633691aa861077263245d328f5963"
server_key_version: "encryptionkey@0"
I20260807 22:44:35.045239 32221 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:44:35.049782 32236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:35.051486 32221 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:44:35.051687 32221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "9b6983bcac0f46d4927da6fdee65791e"
format_stamp: "Formatted at 2026-08-07 22:44:35 on dist-test-slave-csg5"
server_key: "4079d22022999ae1f28ef134aac18c2d"
server_key_iv: "fb1633691aa861077263245d328f5963"
server_key_version: "encryptionkey@0"
I20260807 22:44:35.051925 32221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:35.078163 32221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:35.082978 32221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:35.083330 32221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:35.084445 32221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:35.086189 32221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:35.086297 32221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:35.086445 32221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:35.086519 32221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:35.126084 32221 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:45639
I20260807 22:44:35.126180 32366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:45639 every 8 connection(s)
I20260807 22:44:35.127559 32221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:35.136927 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 32221
I20260807 22:44:35.137162 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:44:35.137826 20522 external_mini_cluster.cc:1428] Setting key 6a53f80a08b3b0cbd8a4db1e80eba607
Aug 07 22:44:35 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:44:35.153028 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.130640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:42523 (local address 127.20.10.190:36673)
0807 22:44:35.131287 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.131292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.131994 (+ 702us) server_negotiation.cc:407] Connection header received
0807 22:44:35.134456 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.134463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.134601 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.134782 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:35.137115 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.138450 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.141064 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.141854 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.141978 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.146333 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:35.146341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:35.146346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:35.146403 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:35.149300 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.150192 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.150196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.150198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.150334 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.150868 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.150872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.150874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.151168 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:35.151302 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.152385 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.152606 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":289,"threads_started":1}
I20260807 22:44:35.154398 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.131046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36673 (local address 127.20.10.131:42523)
0807 22:44:35.131821 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.131864 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.134086 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.135084 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.135097 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.135989 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:35.136837 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.136863 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.138661 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.138690 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.140893 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.140917 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.142100 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.142106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.142385 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.143352 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:35.143380 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:35.146050 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:35.149472 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:35.149481 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:35.149496 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:35.150065 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:35.150449 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:35.150452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:35.150455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:35.150719 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:35.151290 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:35.151296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:35.151691 (+ 395us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.152059 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.152659 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":313,"threads_started":1}
I20260807 22:44:35.156044 32367 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36673
I20260807 22:44:35.156384 32367 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:35.157384 32367 heartbeater.cc:511] Master 127.20.10.190:36673 requested a full tablet report, sending...
I20260807 22:44:35.159310 31855 ts_manager.cc:194] Registered new tserver with Master: 9b6983bcac0f46d4927da6fdee65791e (127.20.10.131:45639)
I20260807 22:44:35.160182 31855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:42523
I20260807 22:44:35.167508 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:44:35.187510 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.172525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.20.10.190:36673)
0807 22:44:35.172721 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.172726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.173112 (+ 386us) server_negotiation.cc:407] Connection header received
0807 22:44:35.173279 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.173283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.173377 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.173477 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:35.175243 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.176128 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.177510 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.177975 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.178088 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.180707 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:35.180713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:35.180716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:35.180792 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:35.183979 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.184888 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.184893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.184896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.185067 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.185734 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.185739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.185741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.186154 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:35.186406 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.186940 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.187191 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:44:35.192427 31855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:44:35.200229 31855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:44:35.229698 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.215989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.20.10.129:34269)
0807 22:44:35.218208 (+ 2219us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.218218 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.218924 (+ 706us) server_negotiation.cc:407] Connection header received
0807 22:44:35.219261 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.219289 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.219574 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.219761 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.221309 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.223268 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.226286 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.227960 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.228083 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.228356 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.229117 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.229317 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2029,"thread_start_us":1861,"threads_started":1}
I20260807 22:44:35.230581 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.215179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:34269 (local address 127.0.0.1:47918)
0807 22:44:35.218788 (+ 3609us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.218810 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.219048 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.220411 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.220417 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.220452 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.220978 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.220997 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.223499 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.223505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.226074 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.226092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.228566 (+ 2474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.228571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.228806 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.228876 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.229955 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.230182 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3398,"thread_start_us":3253,"threads_started":1}
I20260807 22:44:35.231789 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.214796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:45639 (local address 127.0.0.1:39934)
0807 22:44:35.217260 (+ 2464us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.217309 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.217518 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.219958 (+ 2440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.219970 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.220013 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.220536 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.220555 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.224805 (+ 4250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.224812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.227179 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.227197 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.230831 (+ 3634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.230842 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.231044 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.231091 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.231161 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.231327 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2199,"mutex_wait_us":1,"thread_start_us":2038,"threads_started":1}
I20260807 22:44:35.232412 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.215134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.20.10.131:45639)
0807 22:44:35.219224 (+ 4090us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.219233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.219264 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:44:35.219355 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.219362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.219572 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.222568 (+ 2996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.223018 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.224623 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.228985 (+ 4362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.230517 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.231365 (+ 848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.231648 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.231737 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.231938 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3899,"thread_start_us":3760,"threads_started":1}
I20260807 22:44:35.236533 32290 tablet_service.cc:1467] Processing CreateTablet for tablet 67728347ccd24f2ea89055b34a610492 (DEFAULT_TABLE table=test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:35.236995 31989 tablet_service.cc:1467] Processing CreateTablet for tablet 67728347ccd24f2ea89055b34a610492 (DEFAULT_TABLE table=test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:35.238276 32290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67728347ccd24f2ea89055b34a610492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:35.239464 31989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67728347ccd24f2ea89055b34a610492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:35.241729 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.219702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.20.10.130:41827)
0807 22:44:35.233668 (+ 13966us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.233677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.233713 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:44:35.233838 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.233846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.234116 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.234403 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.234855 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.236495 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.238855 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.240507 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.240698 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.240919 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.241169 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.241352 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13763,"thread_start_us":13402,"threads_started":1}
I20260807 22:44:35.241820 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.219098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41827 (local address 127.0.0.1:51422)
0807 22:44:35.231016 (+ 11918us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.231039 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.231157 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.234397 (+ 3240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.234403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.234436 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.234686 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.234699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.236673 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.236681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.238579 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.238598 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.240715 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.240724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.240925 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.240964 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.241081 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.241269 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11797,"thread_start_us":13784,"threads_started":1}
I20260807 22:44:35.246997 32139 tablet_service.cc:1467] Processing CreateTablet for tablet 67728347ccd24f2ea89055b34a610492 (DEFAULT_TABLE table=test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:35.248705 32139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67728347ccd24f2ea89055b34a610492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:35.253612 32388 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Bootstrap starting.
I20260807 22:44:35.256417 32389 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Bootstrap starting.
I20260807 22:44:35.258780 32388 tablet_bootstrap.cc:654] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:35.260241 32388 log.cc:824] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:35.260568 32389 tablet_bootstrap.cc:654] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:35.262013 32389 log.cc:824] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:35.264124 32388 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: No bootstrap required, opened a new log
I20260807 22:44:35.264621 32388 ts_tablet_manager.cc:1397] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 22:44:35.265580 32393 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Bootstrap starting.
I20260807 22:44:35.265740 32389 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: No bootstrap required, opened a new log
I20260807 22:44:35.266085 32389 ts_tablet_manager.cc:1397] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260807 22:44:35.270004 32393 tablet_bootstrap.cc:654] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:35.271533 32393 log.cc:824] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:35.274099 32388 raft_consensus.cc:348] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.274621 32388 raft_consensus.cc:374] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:35.274766 32388 raft_consensus.cc:729] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9711c65ebe6046d0b2ced066ba55277b, State: Initialized, Role: FOLLOWER
I20260807 22:44:35.275478 32388 consensus_queue.cc:260] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.277527 32388 ts_tablet_manager.cc:1428] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 22:44:35.279422 32389 raft_consensus.cc:348] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.279996 32389 raft_consensus.cc:374] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:35.280115 32389 raft_consensus.cc:729] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6983bcac0f46d4927da6fdee65791e, State: Initialized, Role: FOLLOWER
I20260807 22:44:35.280928 32393 tablet_bootstrap.cc:492] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: No bootstrap required, opened a new log
I20260807 22:44:35.280864 32389 consensus_queue.cc:260] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.281330 32393 ts_tablet_manager.cc:1397] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260807 22:44:35.282173 32367 heartbeater.cc:503] Master 127.20.10.190:36673 was elected leader, sending a full tablet report...
I20260807 22:44:35.283064 32389 ts_tablet_manager.cc:1428] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260807 22:44:35.291540 32393 raft_consensus.cc:348] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.292011 32393 raft_consensus.cc:374] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:35.292136 32393 raft_consensus.cc:729] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e64e2221d8e44ec8d00b1d86796811c, State: Initialized, Role: FOLLOWER
I20260807 22:44:35.292795 32393 consensus_queue.cc:260] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.294052 32215 heartbeater.cc:503] Master 127.20.10.190:36673 was elected leader, sending a full tablet report...
I20260807 22:44:35.294582 32393 ts_tablet_manager.cc:1428] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260807 22:44:35.379608 32368 tablet.cc:2369] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:44:35.390777 32064 tablet.cc:2369] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:44:35.513661 32216 tablet.cc:2369] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:44:35.643446 32397 raft_consensus.cc:482] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:44:35.643790 32397 raft_consensus.cc:504] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.644979 32396 raft_consensus.cc:482] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:44:35.645409 32396 raft_consensus.cc:504] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.646035 32397 leader_election.cc:288] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b6983bcac0f46d4927da6fdee65791e (127.20.10.131:45639), 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269)
I20260807 22:44:35.648339 32396 leader_election.cc:288] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269), 6e64e2221d8e44ec8d00b1d86796811c (127.20.10.130:41827)
I20260807 22:44:35.656924 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.646284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:45639 (local address 127.20.10.130:44779)
0807 22:44:35.647726 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.647750 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.648017 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.648685 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.648694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.648728 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.649443 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.649463 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.651289 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.651294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.653364 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.653380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.655823 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.655830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.656045 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.656087 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.656202 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.656382 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"mutex_wait_us":3,"thread_start_us":1022,"threads_started":1}
I20260807 22:44:35.657750 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.646980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:40857 (local address 127.20.10.129:34269)
0807 22:44:35.647248 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.647255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.648205 (+ 950us) server_negotiation.cc:407] Connection header received
0807 22:44:35.648495 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.648501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.648584 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.648660 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.651204 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.652431 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.655120 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.656429 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.656563 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.656649 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.657269 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.657482 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":48}
I20260807 22:44:35.657845 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.646808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44779 (local address 127.20.10.131:45639)
0807 22:44:35.647072 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.647081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.647874 (+ 793us) server_negotiation.cc:407] Connection header received
0807 22:44:35.648204 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.648210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.648316 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.648441 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.649625 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.651102 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.653565 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.655352 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.655542 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.655657 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.657435 (+ 1778us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.657596 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":90}
I20260807 22:44:35.658793 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.646534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:34269 (local address 127.20.10.130:40857)
0807 22:44:35.648103 (+ 1569us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.648125 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.648315 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.648907 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.648915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.648950 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.650634 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.650678 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.652596 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.652602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.654936 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.654951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.656886 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.656892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.657075 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.657108 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.658188 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.658345 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1436,"mutex_wait_us":77,"thread_start_us":1223,"threads_started":1}
I20260807 22:44:35.659017 32314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "6e64e2221d8e44ec8d00b1d86796811c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6983bcac0f46d4927da6fdee65791e" is_pre_election: true
I20260807 22:44:35.659722 32314 raft_consensus.cc:2454] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e64e2221d8e44ec8d00b1d86796811c in term 0.
I20260807 22:44:35.660341 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.648557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:34269 (local address 127.20.10.131:57889)
0807 22:44:35.648767 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.648790 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.648914 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.651663 (+ 2749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.651669 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.651716 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.652000 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.652012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.653729 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.653737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.656246 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.656263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.659390 (+ 3127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.659396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.659638 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.659679 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.659768 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.659944 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":4}
I20260807 22:44:35.660198 32012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "6e64e2221d8e44ec8d00b1d86796811c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9711c65ebe6046d0b2ced066ba55277b" is_pre_election: true
I20260807 22:44:35.660692 32089 leader_election.cc:302] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e64e2221d8e44ec8d00b1d86796811c, 9b6983bcac0f46d4927da6fdee65791e; no voters:
I20260807 22:44:35.660848 32012 raft_consensus.cc:2454] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e64e2221d8e44ec8d00b1d86796811c in term 0.
I20260807 22:44:35.661074 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.650880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:57007 (local address 127.20.10.130:41827)
0807 22:44:35.651121 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.651128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.651148 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:44:35.651208 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.651214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.651289 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.651380 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.654076 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.655245 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.658511 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.659827 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.659984 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.660065 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.660670 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.660812 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 22:44:35.661329 32397 raft_consensus.cc:2781] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:44:35.661500 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.649441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:57889 (local address 127.20.10.129:34269)
0807 22:44:35.651271 (+ 1830us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.651284 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.651310 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:44:35.651376 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.651381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.651459 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.652171 (+ 712us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:35.652491 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.653516 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.656940 (+ 3424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.659174 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.660779 (+ 1605us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.660877 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.660935 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.661089 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":691,"server-negotiator.queue_time_us":1675,"thread_start_us":1416,"threads_started":1}
I20260807 22:44:35.661495 32397 raft_consensus.cc:482] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:44:35.661619 32397 raft_consensus.cc:3037] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:35.661831 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.649029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:41827 (local address 127.20.10.131:57007)
0807 22:44:35.649694 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:35.649713 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:35.649813 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:35.651554 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:35.651561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:35.651591 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:35.653875 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.653889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.655482 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.655487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.657838 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:35.657856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.660242 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.660249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:35.660465 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.660499 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:44:35.661313 (+ 814us) client_negotiation.cc:240] Negotiation successful
0807 22:44:35.661459 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":1739,"thread_start_us":339,"threads_started":1}
I20260807 22:44:35.662120 32012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "9b6983bcac0f46d4927da6fdee65791e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9711c65ebe6046d0b2ced066ba55277b" is_pre_election: true
I20260807 22:44:35.662364 32012 raft_consensus.cc:2454] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6983bcac0f46d4927da6fdee65791e in term 0.
I20260807 22:44:35.662902 32237 leader_election.cc:302] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9711c65ebe6046d0b2ced066ba55277b, 9b6983bcac0f46d4927da6fdee65791e; no voters:
I20260807 22:44:35.662873 32162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "9b6983bcac0f46d4927da6fdee65791e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e64e2221d8e44ec8d00b1d86796811c" is_pre_election: true
I20260807 22:44:35.663374 32396 raft_consensus.cc:2781] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:44:35.663549 32396 raft_consensus.cc:482] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:44:35.663671 32396 raft_consensus.cc:3037] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:35.665457 32397 raft_consensus.cc:504] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.666023 32162 raft_consensus.cc:2379] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b6983bcac0f46d4927da6fdee65791e in current term 1: Already voted for candidate 6e64e2221d8e44ec8d00b1d86796811c in this term.
I20260807 22:44:35.667050 32012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "6e64e2221d8e44ec8d00b1d86796811c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9711c65ebe6046d0b2ced066ba55277b"
I20260807 22:44:35.667057 32314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "6e64e2221d8e44ec8d00b1d86796811c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6983bcac0f46d4927da6fdee65791e"
I20260807 22:44:35.667320 32012 raft_consensus.cc:3037] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:35.667366 32396 raft_consensus.cc:504] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.667917 32314 raft_consensus.cc:2379] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e64e2221d8e44ec8d00b1d86796811c in current term 1: Already voted for candidate 9b6983bcac0f46d4927da6fdee65791e in this term.
I20260807 22:44:35.668256 32396 leader_election.cc:288] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [CANDIDATE]: Term 1 election: Requested vote from peers 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269), 6e64e2221d8e44ec8d00b1d86796811c (127.20.10.130:41827)
I20260807 22:44:35.668466 32397 leader_election.cc:288] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [CANDIDATE]: Term 1 election: Requested vote from peers 9b6983bcac0f46d4927da6fdee65791e (127.20.10.131:45639), 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269)
I20260807 22:44:35.669119 32162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "9b6983bcac0f46d4927da6fdee65791e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e64e2221d8e44ec8d00b1d86796811c"
I20260807 22:44:35.669135 32011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67728347ccd24f2ea89055b34a610492" candidate_uuid: "9b6983bcac0f46d4927da6fdee65791e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9711c65ebe6046d0b2ced066ba55277b"
I20260807 22:44:35.669426 32162 raft_consensus.cc:2379] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b6983bcac0f46d4927da6fdee65791e in current term 1: Already voted for candidate 6e64e2221d8e44ec8d00b1d86796811c in this term.
I20260807 22:44:35.670126 32237 leader_election.cc:302] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b6983bcac0f46d4927da6fdee65791e; no voters: 6e64e2221d8e44ec8d00b1d86796811c, 9711c65ebe6046d0b2ced066ba55277b
I20260807 22:44:35.670469 32396 raft_consensus.cc:2726] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 22:44:35.671346 32012 raft_consensus.cc:2454] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e64e2221d8e44ec8d00b1d86796811c in term 1.
I20260807 22:44:35.671875 32087 leader_election.cc:302] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e64e2221d8e44ec8d00b1d86796811c, 9711c65ebe6046d0b2ced066ba55277b; no voters: 9b6983bcac0f46d4927da6fdee65791e
I20260807 22:44:35.672384 32397 raft_consensus.cc:2781] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:35.672982 32397 raft_consensus.cc:686] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 LEADER]: Becoming Leader. State: Replica: 6e64e2221d8e44ec8d00b1d86796811c, State: Running, Role: LEADER
I20260807 22:44:35.673640 32397 consensus_queue.cc:237] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } }
I20260807 22:44:35.678952 31855 catalog_manager.cc:5495] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e64e2221d8e44ec8d00b1d86796811c (127.20.10.130). New cstate: current_term: 1 leader_uuid: "6e64e2221d8e44ec8d00b1d86796811c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e64e2221d8e44ec8d00b1d86796811c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 41827 } health_report { overall_health: HEALTHY } } }
I20260807 22:44:35.807006 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.800412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.20.10.130:41827)
0807 22:44:35.800643 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.800651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.801129 (+ 478us) server_negotiation.cc:407] Connection header received
0807 22:44:35.801406 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.801411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.801513 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.801686 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:44:35.802233 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.803075 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.804552 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.804976 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.805137 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.805683 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:44:35.806273 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:44:35.806446 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.806560 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.806679 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:44:35.815717 32012 raft_consensus.cc:1261] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 1 FOLLOWER]: Refusing update from remote peer 6e64e2221d8e44ec8d00b1d86796811c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:44:35.815748 32314 raft_consensus.cc:1261] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Refusing update from remote peer 6e64e2221d8e44ec8d00b1d86796811c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:44:35.816679 32403 consensus_queue.cc:1035] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b6983bcac0f46d4927da6fdee65791e" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:44:35.817091 32397 consensus_queue.cc:1035] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9711c65ebe6046d0b2ced066ba55277b" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:44:35.832484 32408 mvcc.cc:205] Tried to move back new op lower bound from 7316040400130162688 to 7316040399563542528. Current Snapshot: MvccSnapshot[applied={T|T < 7316040400130162688}]
I20260807 22:44:35.851421 31855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:44:35.851714 31855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:44:35.855500 31855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 166297ae108a483a876fe886a34db5c3: Sending DeleteTablet for 3 replicas of tablet 67728347ccd24f2ea89055b34a610492
I20260807 22:44:35.856927 32290 tablet_service.cc:1514] Processing DeleteTablet for tablet 67728347ccd24f2ea89055b34a610492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:39934
I20260807 22:44:35.857213 32139 tablet_service.cc:1514] Processing DeleteTablet for tablet 67728347ccd24f2ea89055b34a610492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:51422
I20260807 22:44:35.857981 32420 tablet_replica.cc:330] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: stopping tablet replica
I20260807 22:44:35.858037 31989 tablet_service.cc:1514] Processing DeleteTablet for tablet 67728347ccd24f2ea89055b34a610492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:47918
I20260807 22:44:35.858536 32420 raft_consensus.cc:2229] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:44:35.859220 32420 raft_consensus.cc:2258] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:44:35.859336 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31920
I20260807 22:44:35.861517 32420 ts_tablet_manager.cc:1905] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:44:35.871404 32422 tablet_replica.cc:330] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: stopping tablet replica
I20260807 22:44:35.872176 32422 raft_consensus.cc:2229] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:44:35.873376 32422 raft_consensus.cc:2258] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:44:35.876163 32422 ts_tablet_manager.cc:1905] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:44:35.878367 32421 tablet_replica.cc:330] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: stopping tablet replica
I20260807 22:44:35.878552 32420 ts_tablet_manager.cc:1918] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:44:35.878798 32420 log.cc:1196] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/67728347ccd24f2ea89055b34a610492
I20260807 22:44:35.879405 32420 ts_tablet_manager.cc:1939] T 67728347ccd24f2ea89055b34a610492 P 9b6983bcac0f46d4927da6fdee65791e: Deleting consensus metadata
I20260807 22:44:35.881129 32421 raft_consensus.cc:2229] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:44:35.881548 31837 catalog_manager.cc:4841] TS 9b6983bcac0f46d4927da6fdee65791e (127.20.10.131:45639): tablet 67728347ccd24f2ea89055b34a610492 (table test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]) successfully deleted
I20260807 22:44:35.882217 32421 raft_consensus.cc:2258] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:44:35.884723 32421 ts_tablet_manager.cc:1905] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:44:35 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:44:35.893836 32421 ts_tablet_manager.cc:1918] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:44:35.894146 32421 log.cc:1196] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/67728347ccd24f2ea89055b34a610492
I20260807 22:44:35.894897 32421 ts_tablet_manager.cc:1939] T 67728347ccd24f2ea89055b34a610492 P 6e64e2221d8e44ec8d00b1d86796811c: Deleting consensus metadata
I20260807 22:44:35.896032 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:35.861548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.20.10.129:34269)
0807 22:44:35.861749 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:35.861755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:35.871475 (+ 9720us) server_negotiation.cc:407] Connection header received
0807 22:44:35.872163 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:35.872183 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:35.872286 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:35.872371 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:35.874546 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.875741 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.878616 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:35.879347 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:35.879464 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:35.887209 (+ 7745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:35.887229 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:35.887260 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:35.887311 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:35.891846 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.893066 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.893080 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.893090 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.893305 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:35.893878 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:35.893884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:35.893887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:35.894389 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:35.894691 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:35.895352 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 22:44:35.895657 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:44:35.896981 32422 ts_tablet_manager.cc:1918] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:44:35.897219 32422 log.cc:1196] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/67728347ccd24f2ea89055b34a610492
I20260807 22:44:35.897672 32056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:35.897817 32422 ts_tablet_manager.cc:1939] T 67728347ccd24f2ea89055b34a610492 P 9711c65ebe6046d0b2ced066ba55277b: Deleting consensus metadata
I20260807 22:44:35.899878 31836 catalog_manager.cc:4841] TS 6e64e2221d8e44ec8d00b1d86796811c (127.20.10.130:41827): tablet 67728347ccd24f2ea89055b34a610492 (table test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]) successfully deleted
I20260807 22:44:35.997180 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31920
I20260807 22:44:35.998409 31836 catalog_manager.cc:4841] TS 9711c65ebe6046d0b2ced066ba55277b (127.20.10.129:34269): tablet 67728347ccd24f2ea89055b34a610492 (table test-table [id=5bf333ac164f412a9aacfeafac5c4a7d]) successfully deleted
I20260807 22:44:36.033064 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 32069
Aug 07 22:44:36 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:44:36.050992 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:36.033978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.20.10.130:41827)
0807 22:44:36.034386 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:36.034391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:36.034407 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:44:36.034464 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:36.034468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:36.034543 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:36.034665 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:36.036074 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:36.036875 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:36.038257 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:36.038817 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:36.038960 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:36.043598 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:36.043617 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:36.043634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:36.043702 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:36.047622 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:36.048533 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:36.048542 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:36.048548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:36.048720 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:36.049148 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:36.049153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:36.049158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:36.049672 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:36.050033 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:36.050330 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 22:44:36.050601 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:44:36.051699 32208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:36.177788 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 32069
I20260807 22:44:36.204342 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 32221
Aug 07 22:44:36 dist-test-slave-csg5 krb5kdc[31802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:44:36.222608 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:36.205266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.20.10.131:45639)
0807 22:44:36.205953 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:36.205957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:36.205974 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:44:36.206035 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:36.206039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:36.206183 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:36.206332 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:36.207928 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:36.208749 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:36.210260 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:36.210973 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:36.211163 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:36.216137 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:36.216236 (+ 99us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:36.216254 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:36.216300 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:36.219462 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:36.220273 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:36.220281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:36.220284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:36.220423 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:36.220857 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:36.220862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:36.220873 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:36.221289 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:36.221662 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:36.221972 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 22:44:36.222241 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":298,"threads_started":1}
I20260807 22:44:36.223461 32362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:36.320235 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 32221
I20260807 22:44:36.346251 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 31815
I20260807 22:44:36.347864 31885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:44:36.474817 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 31815
2026-08-07T22:44:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5478 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:44:36 dist-test-slave-csg5 krb5kdc[32431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:44:36 dist-test-slave-csg5 krb5kdc[32431](info): set up 2 sockets
Aug 07 22:44:36 dist-test-slave-csg5 krb5kdc[32431](info): commencing operation
krb5kdc: starting...
W20260807 22:44:37.749430 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:44:37 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangeradmin_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangeradmin_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangerlookup_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangerlookup_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
I20260807 22:44:38.259146 20522 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKr6fn6/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/postgres -l logfile start
2026-08-07 22:44:41.655 UTC [32465] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:44:41.655 UTC [32465] LOG: listening on IPv4 address "127.20.10.148", port 33665
2026-08-07 22:44:41.660 UTC [32465] LOG: listening on Unix socket "/tmp/.s.PGSQL.33665"
2026-08-07 22:44:41.689 UTC [32467] LOG: database system was shut down at 2026-08-07 22:44:39 UTC
2026-08-07 22:44:41.695 UTC [32465] LOG: database system is ready to accept connections
I20260807 22:44:42.790060 20522 mini_postgres.cc:96] Postgres bound to 33665
2026-08-07 22:44:42.819 UTC [32477] FATAL: database "slave" does not exist
127.20.10.148:33665 - accepting connections
I20260807 22:44:42.820103 20522 mini_ranger.cc:161] Starting Ranger...
I20260807 22:44:42.858088 20522 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:44:43.596558 20522 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:44:43.596710 20522 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin
2026-08-07 22:44:43,921 [I] DB FLAVOR :POSTGRES
2026-08-07 22:44:43,922 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:44:43,922 [I] Checking connection..
2026-08-07 22:44:43,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:44:44,217 [I] Checking connection passed.
2026-08-07 22:44:44,217 [I] --------- Verifying version history table ---------
2026-08-07 22:44:44,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:44,515 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:44:44,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:44,815 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:44:44,816 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:44:44,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:44:45.123 UTC [32546] WARNING: there is no transaction in progress
2026-08-07 22:44:45,132 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:44:45,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:45,409 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:44:45,409 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:44:45,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:44:45,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:44:45,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-csg5.c.gcp-upstream.internal','N') ;"
2026-08-07 22:44:46,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:44:46,559 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:44:46,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:44:46,844 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:44:46,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:44:47,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:44:47,440 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:44:47,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:44:48.655 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.675 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.695 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.715 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.733 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.754 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.821 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.829 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.848 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.860 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.874 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.892 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.905 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.914 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.924 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:48.933 UTC [32664] WARNING: there is no transaction in progress
2026-08-07 22:44:49,892 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:44:49,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:44:50,170 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:44:50,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:44:50,463 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:44:50,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:44:50,762 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:44:50,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:44:51,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:44:51,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-csg5.c.gcp-upstream.internal';"
2026-08-07 22:44:51,628 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:44:51,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:44:51,931 [I] DB_PATCHES have already been applied
I20260807 22:44:51.938393 20522 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKr6fn6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:44:51.938529 20522 mini_ranger.cc:193] Using host: 127.20.10.148
I20260807 22:44:51.965021 20522 mini_ranger.cc:227] Ranger admin URL: http://127.20.10.148:44951
Aug 07, 2026 10:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:44:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:44:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:44:53.324936 20522 mini_ranger.cc:160] Time spent starting Ranger: real 10.505s user 0.000s sys 0.081s
Aug 07, 2026 10:45:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:45:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 10:45:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:45:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:45:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:45:17.960295 20522 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangerkms_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/rangerkms_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.10.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.10.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:45:18 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:45:18.254033 20522 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:45:18.291551 20522 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:45:19.471187 20522 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:45:19.471319 20522 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms
I20260807 22:45:19.471490 20522 mini_ranger_kms.cc:227] Using postgres at 127.20.10.148:33665
2026-08-07 22:45:20,038 [I] DB FLAVOR :POSTGRES
2026-08-07 22:45:20,038 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:45:20,038 [I] Checking connection
2026-08-07 22:45:20,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:45:20,323 [I] connection success
2026-08-07 22:45:20,323 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:45:20,323 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:45:20,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:45:20,619 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:45:20,620 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:45:20,620 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:45:20,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:45:20,946 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:45:21.563702 20522 mini_ranger_kms.cc:310] Created kms service
I20260807 22:45:21.814445 20522 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:45:21.872951 20522 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:45:21 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:22.424063 20522 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:45:22.425833 20522 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:45:22.425912 20522 mini_ranger_kms.cc:240] Using host: 127.20.10.148
I20260807 22:45:22.457419 20522 mini_ranger_kms.cc:276] Ranger KMS PID: 391
I20260807 22:45:22.457654 20522 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.10.148:52549
Aug 07, 2026 10:45:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:45:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:45:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:45:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:45:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:45:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 22:45:25.194723 20522 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.941s user 0.153s sys 0.463s
I20260807 22:45:25.195106 20522 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:45:25.195361 20522 mini_ranger_kms.cc:401] 127.20.10.148:52549/kms/v1/keys
Aug 07 22:45:25 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142725, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 22:45:25 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142725, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
Aug 07 22:45:26 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142725, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
Aug 07 22:45:28 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:45:28.995942 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34693
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:34693
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:52549/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:29.652758 447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:29.653244 447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:29.653532 447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:29.665366 447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:29.665627 447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:29.665766 447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:29.665989 447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:29.666168 447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:29.681752 447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.10.148:52549/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:34693
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:45:29.684533 447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:29.686647 447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:29.708503 453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:29.708626 454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:29.709952 456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:29.710875 447 server_base.cc:1034] running on GCE node
I20260807 22:45:29.711892 447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:29.713843 447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:29.715278 447 hybrid_clock.cc:648] HybridClock initialized: now 1786142729715165 us; error 55 us; skew 500 ppm
I20260807 22:45:29.715835 447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:45:29 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:29.726016 447 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:45:29.728916 447 webserver.cc:460] Webserver started at http://127.20.10.190:35149/ using document root <none> and password file <none>
I20260807 22:45:29.729964 447 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:29.730268 447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:29.730831 447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:29 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:29.804255 447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "05babdacd38e4ea4a11d95c72403b2e6"
format_stamp: "Formatted at 2026-08-07 22:45:29 on dist-test-slave-csg5"
server_key: "f3c7d50473a46ef9d4f697f4503b5b7e"
server_key_iv: "9a5ae782c64ce72caaf14b0c2214dcb5"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:29.805536 447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "05babdacd38e4ea4a11d95c72403b2e6"
format_stamp: "Formatted at 2026-08-07 22:45:29 on dist-test-slave-csg5"
server_key: "f3c7d50473a46ef9d4f697f4503b5b7e"
server_key_iv: "9a5ae782c64ce72caaf14b0c2214dcb5"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:29.815124 447 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.005s
I20260807 22:45:29.840361 463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:29.842258 447 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:45:29.842522 447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "05babdacd38e4ea4a11d95c72403b2e6"
format_stamp: "Formatted at 2026-08-07 22:45:29 on dist-test-slave-csg5"
server_key: "f3c7d50473a46ef9d4f697f4503b5b7e"
server_key_iv: "9a5ae782c64ce72caaf14b0c2214dcb5"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:29.842878 447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:29.861331 447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:29.868495 447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:29.869067 447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:29.937251 447 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:34693
I20260807 22:45:29.937290 517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:34693 every 8 connection(s)
I20260807 22:45:29.939649 447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:29.941545 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 447
I20260807 22:45:29.941977 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:45:29.943377 20522 external_mini_cluster.cc:1428] Setting key d9edff2e598e44d3fedcbdde7a117154
I20260807 22:45:29.959170 518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:45:29 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:45:29.989239 518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: Bootstrap starting.
I20260807 22:45:29.994028 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:29.946511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.20.10.190:34693)
0807 22:45:29.961901 (+ 15390us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:29.961929 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:29.963348 (+ 1419us) server_negotiation.cc:407] Connection header received
0807 22:45:29.969580 (+ 6232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:29.969713 (+ 133us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:29.970570 (+ 857us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:29.971262 (+ 692us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:29.973132 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:29.975668 (+ 2536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:29.977114 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:29.978120 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:29.978370 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:29.983675 (+ 5305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:29.983687 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:29.983707 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:29.983759 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:29.988872 (+ 5113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:29.989824 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:29.989838 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:29.989845 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:29.990058 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:29.990515 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:29.990520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:29.990524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:29.991247 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:29.991640 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:29.992166 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 22:45:29.992693 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14802,"thread_start_us":16300,"threads_started":1}
I20260807 22:45:29.996331 518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:29.999034 518 log.cc:824] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:30.004928 518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: No bootstrap required, opened a new log
I20260807 22:45:30.023360 518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } }
I20260807 22:45:30.024091 518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:30.024336 518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05babdacd38e4ea4a11d95c72403b2e6, State: Initialized, Role: FOLLOWER
I20260807 22:45:30.025298 518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } }
I20260807 22:45:30.025674 518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:30.025873 518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:30.026096 518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:30.030967 518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } }
I20260807 22:45:30.031975 518 leader_election.cc:302] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05babdacd38e4ea4a11d95c72403b2e6; no voters:
I20260807 22:45:30.033133 518 leader_election.cc:288] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:45:30.033321 523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:30.035149 523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [term 1 LEADER]: Becoming Leader. State: Replica: 05babdacd38e4ea4a11d95c72403b2e6, State: Running, Role: LEADER
I20260807 22:45:30.036216 518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:30.036130 523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } }
I20260807 22:45:30.040616 525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05babdacd38e4ea4a11d95c72403b2e6. Latest consensus state: current_term: 1 leader_uuid: "05babdacd38e4ea4a11d95c72403b2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } } }
I20260807 22:45:30.048843 525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:30.039260 524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05babdacd38e4ea4a11d95c72403b2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05babdacd38e4ea4a11d95c72403b2e6" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 34693 } } }
I20260807 22:45:30.050004 524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:30.061460 531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:30.070497 531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:30.097553 531 catalog_manager.cc:1347] Generated new cluster ID: 764feff81fbc4b76ab94834bfcf151df
I20260807 22:45:30.097851 531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:30.116822 518 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:45:30.129705 531 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:30.147917 531 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:30.175800 531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: Generated new TSK 0
I20260807 22:45:30.177052 531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:45:31.232576 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:52549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:31 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:45:31.968463 567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:31.968870 567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:31.969211 567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:31.981666 567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:31.981982 567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:31.982280 567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:45:32.003335 567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.10.148:52549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:45:32.005681 567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:32.008314 567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:32.037019 582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:32.037588 581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:32.048666 567 server_base.cc:1034] running on GCE node
W20260807 22:45:32.049288 584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:32.050644 567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:32.052409 567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:32.053850 567 hybrid_clock.cc:648] HybridClock initialized: now 1786142732053636 us; error 121 us; skew 500 ppm
I20260807 22:45:32.054491 567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:45:32 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:32.061829 567 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:45:32.070178 567 webserver.cc:460] Webserver started at http://127.20.10.129:45121/ using document root <none> and password file <none>
I20260807 22:45:32.071626 567 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:32.071952 567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:32.072502 567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:32 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:32.099638 567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "48a8b21da5cc44bea3a3518f07a48c3a"
format_stamp: "Formatted at 2026-08-07 22:45:32 on dist-test-slave-csg5"
server_key: "f9cc57b064b4253caad2bcb9d041262e"
server_key_iv: "0a623454676bf5e83754207a9a8e842e"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:32.101832 567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "48a8b21da5cc44bea3a3518f07a48c3a"
format_stamp: "Formatted at 2026-08-07 22:45:32 on dist-test-slave-csg5"
server_key: "f9cc57b064b4253caad2bcb9d041262e"
server_key_iv: "0a623454676bf5e83754207a9a8e842e"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:32.126331 567 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.015s sys 0.007s
I20260807 22:45:32.152426 591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:32.154919 567 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260807 22:45:32.155224 567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "48a8b21da5cc44bea3a3518f07a48c3a"
format_stamp: "Formatted at 2026-08-07 22:45:32 on dist-test-slave-csg5"
server_key: "f9cc57b064b4253caad2bcb9d041262e"
server_key_iv: "0a623454676bf5e83754207a9a8e842e"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:32.155619 567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:32.247889 567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:32.255895 567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:32.256716 567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:32.265743 567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 07 22:45:32 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:32.275663 567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:32.275959 567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 22:45:32.276362 567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:32.276523 567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:32.619774 567 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:37615
I20260807 22:45:32.619834 713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:37615 every 8 connection(s)
I20260807 22:45:32.621891 567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:32.625118 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 567
I20260807 22:45:32.625382 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:45:32.626317 20522 external_mini_cluster.cc:1428] Setting key d3e67d9a4e9e0f1680f89693fa6b0c04
Aug 07 22:45:32 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
I20260807 22:45:32.701889 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:32.631015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:54921 (local address 127.20.10.190:34693)
0807 22:45:32.663984 (+ 32969us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:32.663992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:32.665256 (+ 1264us) server_negotiation.cc:407] Connection header received
0807 22:45:32.673308 (+ 8052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:32.673316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:32.673440 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:32.673556 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:32.676970 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:32.678172 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:32.681517 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:32.682248 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:32.682381 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:32.687944 (+ 5563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:32.687953 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:32.687958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:32.688011 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:32.693410 (+ 5399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:32.698618 (+ 5208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:32.698626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:32.698629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:32.698877 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:32.699555 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:32.699560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:32.699563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:32.699955 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:32.700127 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:32.701224 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0807 22:45:32.701504 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32781,"thread_start_us":32992,"threads_started":1}
I20260807 22:45:32.704533 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:32.629956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34693 (local address 127.20.10.129:54921)
0807 22:45:32.664891 (+ 34935us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:32.664962 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:32.668310 (+ 3348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:32.674339 (+ 6029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:32.674356 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:32.675277 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:32.676622 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:32.676662 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:32.678599 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:32.678606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:32.681233 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:32.681259 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:32.682925 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:32.682935 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:32.683295 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:32.684283 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:32.684318 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:32.687532 (+ 3214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:32.693731 (+ 6199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:32.693743 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:32.693777 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:32.698379 (+ 4602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:32.699069 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:32.699074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:32.699077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:32.699409 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:32.700253 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:32.700262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:32.700759 (+ 497us) client_negotiation.cc:769] Sending connection context
0807 22:45:32.702136 (+ 1377us) client_negotiation.cc:240] Negotiation successful
0807 22:45:32.702825 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34418,"thread_start_us":38646,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:45:32.707201 714 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34693
I20260807 22:45:32.707633 714 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:32.708707 714 heartbeater.cc:511] Master 127.20.10.190:34693 requested a full tablet report, sending...
I20260807 22:45:32.716351 481 ts_manager.cc:194] Registered new tserver with Master: 48a8b21da5cc44bea3a3518f07a48c3a (127.20.10.129:37615)
I20260807 22:45:32.718904 481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:54921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
W20260807 22:45:32.758895 20522 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.20.10.130: real 0.127s user 0.000s sys 0.031s
I20260807 22:45:32.759939 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:52549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:32 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
W20260807 22:45:33.600435 721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:33.601047 721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:33.601390 721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:33.615211 721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:33.615526 721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:33.615842 721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:45:33.636878 721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.10.148:52549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:45:33.639006 721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:33.640749 721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:33.663972 735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:33.665042 738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:33.665138 736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:33.667385 721 server_base.cc:1034] running on GCE node
I20260807 22:45:33.668241 721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:33.669725 721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:33.671027 721 hybrid_clock.cc:648] HybridClock initialized: now 1786142733670910 us; error 78 us; skew 500 ppm
I20260807 22:45:33.671543 721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:45:33 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142733, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:33.677896 721 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:45:33.681288 721 webserver.cc:460] Webserver started at http://127.20.10.130:37797/ using document root <none> and password file <none>
I20260807 22:45:33.682926 721 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:33.683251 721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:33.683766 721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:33 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142733, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:33.705677 721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "797283d0cdab4f859c15a703152ad517"
format_stamp: "Formatted at 2026-08-07 22:45:33 on dist-test-slave-csg5"
server_key: "ddd0b4ebf024932be90cc3e753d59b46"
server_key_iv: "18c4f8ccff2fcf06e5597f9684bae82f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:33.707262 721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "797283d0cdab4f859c15a703152ad517"
format_stamp: "Formatted at 2026-08-07 22:45:33 on dist-test-slave-csg5"
server_key: "ddd0b4ebf024932be90cc3e753d59b46"
server_key_iv: "18c4f8ccff2fcf06e5597f9684bae82f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:33.716874 721 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.006s
I20260807 22:45:33.723933 714 heartbeater.cc:503] Master 127.20.10.190:34693 was elected leader, sending a full tablet report...
I20260807 22:45:33.741446 745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:33.743955 721 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260807 22:45:33.744431 721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "797283d0cdab4f859c15a703152ad517"
format_stamp: "Formatted at 2026-08-07 22:45:33 on dist-test-slave-csg5"
server_key: "ddd0b4ebf024932be90cc3e753d59b46"
server_key_iv: "18c4f8ccff2fcf06e5597f9684bae82f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:33.744911 721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:33.772063 721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:33.782025 721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:33.782856 721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:33.785686 721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:33.788944 721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:33.789072 721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:33.789227 721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:33.789316 721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:33.927505 721 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:33845
I20260807 22:45:33.927620 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:33845 every 8 connection(s)
I20260807 22:45:33.929912 721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:33.940281 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 721
I20260807 22:45:33.940481 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:45:33.941120 20522 external_mini_cluster.cc:1428] Setting key f7fa9ec1da0eb901c326e9cd79ffb16c
Aug 07 22:45:33 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142733, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
I20260807 22:45:33.988664 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:33.935496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:53831 (local address 127.20.10.190:34693)
0807 22:45:33.950194 (+ 14698us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:33.950204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:33.957146 (+ 6942us) server_negotiation.cc:407] Connection header received
0807 22:45:33.961945 (+ 4799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:33.961952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:33.962083 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:33.962247 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:33.965464 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:33.967201 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:33.970053 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:33.970895 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:33.971084 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:33.977250 (+ 6166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:33.977259 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:33.977263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:33.977310 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:33.983056 (+ 5746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:33.984368 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:33.984376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:33.984380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:33.984658 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:33.985430 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:33.985434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:33.985436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:33.986002 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:33.986240 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:33.987648 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0807 22:45:33.987963 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14461,"thread_start_us":14157,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
I20260807 22:45:33.990329 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:33.936868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34693 (local address 127.20.10.130:53831)
0807 22:45:33.956917 (+ 20049us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:33.956990 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:33.961505 (+ 4515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:33.962640 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:33.962714 (+ 74us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:33.963808 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:33.965171 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:33.965203 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:33.967497 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:33.967502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:33.969811 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:33.969832 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:33.971448 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:33.971460 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:33.971849 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:33.972960 (+ 1111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:33.973035 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:33.976853 (+ 3818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:33.983429 (+ 6576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:33.983438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:33.983456 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:33.984155 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:33.984829 (+ 674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:33.984834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:33.984838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:33.985227 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:33.986203 (+ 976us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:33.986213 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:33.986980 (+ 767us) client_negotiation.cc:769] Sending connection context
0807 22:45:33.987525 (+ 545us) client_negotiation.cc:240] Negotiation successful
0807 22:45:33.988318 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19543,"thread_start_us":19074,"threads_started":1}
I20260807 22:45:33.992136 866 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34693
I20260807 22:45:33.992487 866 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:33.993292 866 heartbeater.cc:511] Master 127.20.10.190:34693 requested a full tablet report, sending...
I20260807 22:45:33.995208 481 ts_manager.cc:194] Registered new tserver with Master: 797283d0cdab4f859c15a703152ad517 (127.20.10.130:33845)
I20260807 22:45:33.996299 481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:53831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:45:34.045274 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:52549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:34.828784 872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:34.829190 872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:34.829329 872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:34.841051 872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:34.841331 872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:34.841501 872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:45:34.857097 872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.10.148:52549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:45:34.858834 872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:34.860646 872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:34.882598 878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:34.883625 881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:34.883657 879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:34.889591 872 server_base.cc:1034] running on GCE node
I20260807 22:45:34.890513 872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:34.892450 872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:34.893815 872 hybrid_clock.cc:648] HybridClock initialized: now 1786142734893689 us; error 83 us; skew 500 ppm
I20260807 22:45:34.894397 872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:45:34 dist-test-slave-csg5 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:34.905476 872 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:45:34.908577 872 webserver.cc:460] Webserver started at http://127.20.10.131:36643/ using document root <none> and password file <none>
I20260807 22:45:34.909462 872 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:34.909735 872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:34.910250 872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:34 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:45:34.930644 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "1dcb68a05986444b953fac9be1da2ea4"
format_stamp: "Formatted at 2026-08-07 22:45:34 on dist-test-slave-csg5"
server_key: "8680419268c665dcbafa3440078cfe20"
server_key_iv: "3cf1034ec1593c990f40e904a3b6f131"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:34.931872 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "1dcb68a05986444b953fac9be1da2ea4"
format_stamp: "Formatted at 2026-08-07 22:45:34 on dist-test-slave-csg5"
server_key: "8680419268c665dcbafa3440078cfe20"
server_key_iv: "3cf1034ec1593c990f40e904a3b6f131"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:34.940672 872 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.003s
I20260807 22:45:34.964570 888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:34.967053 872 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:45:34.967384 872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "1dcb68a05986444b953fac9be1da2ea4"
format_stamp: "Formatted at 2026-08-07 22:45:34 on dist-test-slave-csg5"
server_key: "8680419268c665dcbafa3440078cfe20"
server_key_iv: "3cf1034ec1593c990f40e904a3b6f131"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:34.967810 872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:34.999486 866 heartbeater.cc:503] Master 127.20.10.190:34693 was elected leader, sending a full tablet report...
I20260807 22:45:34.999722 872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:35.008320 872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:35.009091 872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:35.011755 872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:35.015022 872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:35.015202 872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:35.015506 872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:35.015663 872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:35.144815 872 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:45435
I20260807 22:45:35.144959 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:45435 every 8 connection(s)
I20260807 22:45:35.146488 872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:35.153954 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 872
I20260807 22:45:35.154160 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:45:35.154971 20522 external_mini_cluster.cc:1428] Setting key acaa6bb842ec4ff690d01e6a2da6d40a
Aug 07 22:45:35 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:45:35.182008 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.151173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:41379 (local address 127.20.10.190:34693)
0807 22:45:35.152764 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.152772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.154919 (+ 2147us) server_negotiation.cc:407] Connection header received
0807 22:45:35.158786 (+ 3867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.158790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.158868 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.158999 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:35.161761 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.162834 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.165355 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.165991 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.166171 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.172514 (+ 6343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:35.172522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:35.172527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:35.172636 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:35.177492 (+ 4856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.178811 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.178818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.178821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.179011 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.179633 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.179639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.179642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.180090 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:35.180254 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.181208 (+ 954us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.181488 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1095,"threads_started":1}
I20260807 22:45:35.183800 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.152034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:34693 (local address 127.20.10.131:41379)
0807 22:45:35.154153 (+ 2119us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.154218 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.158245 (+ 4027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.159486 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.159501 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.160441 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:35.161536 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.161562 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.163032 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.163045 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.165118 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.165151 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.166474 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.166486 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.166894 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.167973 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:35.168001 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:35.171830 (+ 3829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:35.177775 (+ 5945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:35.177784 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:35.177802 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:35.178580 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:35.179171 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:35.179176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:35.179180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:35.179476 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:35.180277 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:35.180285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:35.180769 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.181054 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.181938 (+ 884us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"thread_start_us":868,"threads_started":1}
I20260807 22:45:35.186067 1011 heartbeater.cc:347] Connected to a master server at 127.20.10.190:34693
I20260807 22:45:35.186514 1011 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:35.187770 1011 heartbeater.cc:511] Master 127.20.10.190:34693 requested a full tablet report, sending...
I20260807 22:45:35.189859 479 ts_manager.cc:194] Registered new tserver with Master: 1dcb68a05986444b953fac9be1da2ea4 (127.20.10.131:45435)
I20260807 22:45:35.191284 479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:41379
I20260807 22:45:35.200774 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:35.237242 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.209225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.20.10.190:34693)
0807 22:45:35.209473 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.209480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.211201 (+ 1721us) server_negotiation.cc:407] Connection header received
0807 22:45:35.211432 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.211438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.211543 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.211652 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:35.215634 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.216580 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.219990 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.220616 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.220808 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.226617 (+ 5809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:35.226624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:35.226628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:35.226801 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:35.232693 (+ 5892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.233924 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.233931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.233935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.234251 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.234911 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.234917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.234920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.235458 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:35.235803 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.236234 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.236683 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:45:35.243552 479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:35.257624 479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:35.314808 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.282570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:45435 (local address 127.0.0.1:48346)
0807 22:45:35.283922 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.283979 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.284200 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.300086 (+ 15886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.300099 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.300198 (+ 99us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:35.301050 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.301072 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.305794 (+ 4722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.305801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.309535 (+ 3734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.309550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.313205 (+ 3655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.313215 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.313501 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.313562 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.313723 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.313988 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":10,"thread_start_us":2081,"threads_started":1}
I20260807 22:45:35.315236 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.285528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:33845 (local address 127.0.0.1:40434)
0807 22:45:35.299891 (+ 14363us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.299909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.300023 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.306184 (+ 6161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.306189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.306222 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:35.308132 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.308149 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.310429 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.310435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.312232 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.312248 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.314452 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.314457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.314598 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.314624 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.314768 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.314898 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14189,"mutex_wait_us":30,"thread_start_us":19088,"threads_started":1}
I20260807 22:45:35.316429 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.286078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.20.10.130:33845)
0807 22:45:35.304979 (+ 18901us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.304990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.305033 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:45:35.305172 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.305184 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.305464 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.305801 (+ 337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:35.308505 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.310262 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.312425 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.313879 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.314032 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.314288 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.315818 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.316034 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18648,"thread_start_us":18070,"threads_started":1}
I20260807 22:45:35.316926 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.283311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.20.10.131:45435)
0807 22:45:35.298193 (+ 14882us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.298202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.298244 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:45:35.298376 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.298385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.298766 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.299047 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:35.301547 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.304892 (+ 3345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.309782 (+ 4890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.312268 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.312470 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.312897 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.315877 (+ 2980us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.316117 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14670,"thread_start_us":16762,"threads_started":1}
I20260807 22:45:35.317272 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.289930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.20.10.129:37615)
0807 22:45:35.295284 (+ 5354us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.295296 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.305465 (+ 10169us) server_negotiation.cc:407] Connection header received
0807 22:45:35.305615 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.305623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.305890 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.306131 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:35.308129 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.310059 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.313913 (+ 3854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.315698 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.315853 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.316093 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.316570 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.316867 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5130,"thread_start_us":4947,"threads_started":1}
I20260807 22:45:35.317487 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.289499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37615 (local address 127.0.0.1:58968)
0807 22:45:35.305032 (+ 15533us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.305056 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.305256 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.306417 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.306426 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.306462 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:35.307793 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.307808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.310234 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.310240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.313234 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.313250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.315938 (+ 2688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.315946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.316217 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.316258 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.316354 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.316888 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15362,"mutex_wait_us":80,"thread_start_us":10014,"threads_started":1}
I20260807 22:45:35.321070 939 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb0db2be750491daf259317bdde4799 (DEFAULT_TABLE table=test-table [id=fc198df09a21404cb07f0c91ce85ba03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:35.321192 798 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb0db2be750491daf259317bdde4799 (DEFAULT_TABLE table=test-table [id=fc198df09a21404cb07f0c91ce85ba03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:35.322854 939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb0db2be750491daf259317bdde4799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:35.323007 798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb0db2be750491daf259317bdde4799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:35.324973 641 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb0db2be750491daf259317bdde4799 (DEFAULT_TABLE table=test-table [id=fc198df09a21404cb07f0c91ce85ba03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:35.326692 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb0db2be750491daf259317bdde4799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:35.336735 1031 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Bootstrap starting.
I20260807 22:45:35.339507 1032 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Bootstrap starting.
I20260807 22:45:35.341451 1031 tablet_bootstrap.cc:654] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:35.343052 1031 log.cc:824] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:35.343665 1033 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Bootstrap starting.
I20260807 22:45:35.344063 1032 tablet_bootstrap.cc:654] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:35.345372 1032 log.cc:824] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:35.348562 1033 tablet_bootstrap.cc:654] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:35.349080 1031 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: No bootstrap required, opened a new log
I20260807 22:45:35.349373 1032 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: No bootstrap required, opened a new log
I20260807 22:45:35.349428 1031 ts_tablet_manager.cc:1397] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20260807 22:45:35.349678 1032 ts_tablet_manager.cc:1397] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 22:45:35.350104 1033 log.cc:824] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:35.354115 1033 tablet_bootstrap.cc:492] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: No bootstrap required, opened a new log
I20260807 22:45:35.354506 1033 ts_tablet_manager.cc:1397] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 22:45:35.361869 1031 raft_consensus.cc:348] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.363009 1031 raft_consensus.cc:374] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:35.363448 1031 raft_consensus.cc:729] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 797283d0cdab4f859c15a703152ad517, State: Initialized, Role: FOLLOWER
I20260807 22:45:35.364928 1031 consensus_queue.cc:260] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.366402 1032 raft_consensus.cc:348] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.367264 1032 raft_consensus.cc:374] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:35.367465 1032 raft_consensus.cc:729] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcb68a05986444b953fac9be1da2ea4, State: Initialized, Role: FOLLOWER
I20260807 22:45:35.367595 1031 ts_tablet_manager.cc:1428] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Time spent starting tablet: real 0.018s user 0.015s sys 0.002s
I20260807 22:45:35.368762 1032 consensus_queue.cc:260] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.370692 1011 heartbeater.cc:503] Master 127.20.10.190:34693 was elected leader, sending a full tablet report...
I20260807 22:45:35.371580 1032 ts_tablet_manager.cc:1428] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Time spent starting tablet: real 0.022s user 0.017s sys 0.005s
I20260807 22:45:35.372004 1033 raft_consensus.cc:348] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.372597 1033 raft_consensus.cc:374] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:35.372767 1033 raft_consensus.cc:729] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a8b21da5cc44bea3a3518f07a48c3a, State: Initialized, Role: FOLLOWER
I20260807 22:45:35.373528 1033 consensus_queue.cc:260] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.376232 1033 ts_tablet_manager.cc:1428] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Time spent starting tablet: real 0.022s user 0.018s sys 0.005s
W20260807 22:45:35.400035 1012 tablet.cc:2369] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:45:35.425570 715 tablet.cc:2369] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:45:35.435093 867 tablet.cc:2369] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:35.542642 1038 raft_consensus.cc:482] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:35.543120 1038 raft_consensus.cc:504] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.545354 1038 leader_election.cc:288] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 797283d0cdab4f859c15a703152ad517 (127.20.10.130:33845), 48a8b21da5cc44bea3a3518f07a48c3a (127.20.10.129:37615)
I20260807 22:45:35.553431 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.545685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:33845 (local address 127.20.10.131:47103)
0807 22:45:35.546088 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.546114 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.546402 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.547074 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.547079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.547106 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:35.547535 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.547546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.548867 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.548872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.550943 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.550957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.552273 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.552279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.552613 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.552657 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.552802 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.552951 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260807 22:45:35.553453 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.546387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:47103 (local address 127.20.10.130:33845)
0807 22:45:35.546652 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.546684 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.546711 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:45:35.546813 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.546821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.546912 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.547058 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:35.547696 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.548702 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.551112 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.552058 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.552289 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.552349 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.552859 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.553051 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 22:45:35.555043 818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb0db2be750491daf259317bdde4799" candidate_uuid: "1dcb68a05986444b953fac9be1da2ea4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "797283d0cdab4f859c15a703152ad517" is_pre_election: true
I20260807 22:45:35.555284 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.546377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:45173 (local address 127.20.10.129:37615)
0807 22:45:35.546703 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.546711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.547309 (+ 598us) server_negotiation.cc:407] Connection header received
0807 22:45:35.547500 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.547503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.547588 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.547681 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:35.548799 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.549775 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.552184 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.553581 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.554039 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.554136 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.554788 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.554968 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:45:35.555538 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.546253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:37615 (local address 127.20.10.131:45173)
0807 22:45:35.547191 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.547215 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:35.547351 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:35.547752 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:35.547758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:35.547786 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:35.548502 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.548514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.549976 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.549979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.552005 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:35.552019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.554141 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.554150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:35.554513 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.554540 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:45:35.554700 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:45:35.554862 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":574,"threads_started":1}
I20260807 22:45:35.555764 818 raft_consensus.cc:2454] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcb68a05986444b953fac9be1da2ea4 in term 0.
I20260807 22:45:35.556777 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb0db2be750491daf259317bdde4799" candidate_uuid: "1dcb68a05986444b953fac9be1da2ea4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" is_pre_election: true
I20260807 22:45:35.557278 662 raft_consensus.cc:2454] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcb68a05986444b953fac9be1da2ea4 in term 0.
I20260807 22:45:35.557688 892 leader_election.cc:302] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcb68a05986444b953fac9be1da2ea4, 797283d0cdab4f859c15a703152ad517; no voters:
I20260807 22:45:35.558535 1038 raft_consensus.cc:2781] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:35.558930 1038 raft_consensus.cc:482] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:35.559139 1038 raft_consensus.cc:3037] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:35.563203 1038 raft_consensus.cc:504] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.564208 1038 leader_election.cc:288] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [CANDIDATE]: Term 1 election: Requested vote from peers 797283d0cdab4f859c15a703152ad517 (127.20.10.130:33845), 48a8b21da5cc44bea3a3518f07a48c3a (127.20.10.129:37615)
I20260807 22:45:35.564988 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb0db2be750491daf259317bdde4799" candidate_uuid: "1dcb68a05986444b953fac9be1da2ea4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a8b21da5cc44bea3a3518f07a48c3a"
I20260807 22:45:35.565150 818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb0db2be750491daf259317bdde4799" candidate_uuid: "1dcb68a05986444b953fac9be1da2ea4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "797283d0cdab4f859c15a703152ad517"
I20260807 22:45:35.565294 662 raft_consensus.cc:3037] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:35.565555 818 raft_consensus.cc:3037] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:35.568760 662 raft_consensus.cc:2454] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcb68a05986444b953fac9be1da2ea4 in term 1.
I20260807 22:45:35.569252 818 raft_consensus.cc:2454] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcb68a05986444b953fac9be1da2ea4 in term 1.
I20260807 22:45:35.569478 891 leader_election.cc:302] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcb68a05986444b953fac9be1da2ea4, 48a8b21da5cc44bea3a3518f07a48c3a; no voters:
I20260807 22:45:35.570031 1038 raft_consensus.cc:2781] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:35.571272 1038 raft_consensus.cc:686] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcb68a05986444b953fac9be1da2ea4, State: Running, Role: LEADER
I20260807 22:45:35.571978 1038 consensus_queue.cc:237] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } }
I20260807 22:45:35.578907 481 catalog_manager.cc:5495] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcb68a05986444b953fac9be1da2ea4 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "1dcb68a05986444b953fac9be1da2ea4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcb68a05986444b953fac9be1da2ea4" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 45435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 } health_report { overall_health: UNKNOWN } } }
I20260807 22:45:35.669992 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.662538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.20.10.131:45435)
0807 22:45:35.662967 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.662972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.662996 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:45:35.663118 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.663122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.663312 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.663508 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:35.664310 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.665229 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.667473 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.667973 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.668120 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.668787 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:35.669342 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:35.669507 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.669571 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.669672 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260807 22:45:35.682207 818 raft_consensus.cc:1261] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 1 FOLLOWER]: Refusing update from remote peer 1dcb68a05986444b953fac9be1da2ea4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:35.682246 662 raft_consensus.cc:1261] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 1 FOLLOWER]: Refusing update from remote peer 1dcb68a05986444b953fac9be1da2ea4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:35.684064 1038 consensus_queue.cc:1035] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48a8b21da5cc44bea3a3518f07a48c3a" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:35.684547 1041 consensus_queue.cc:1035] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "797283d0cdab4f859c15a703152ad517" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 33845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:35.703313 1046 mvcc.cc:205] Tried to move back new op lower bound from 7316040645340549120 to 7316040644908244992. Current Snapshot: MvccSnapshot[applied={T|T < 7316040645340549120}]
I20260807 22:45:35.732999 479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:35.733417 479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:35.738720 479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05babdacd38e4ea4a11d95c72403b2e6: Sending DeleteTablet for 3 replicas of tablet 5eb0db2be750491daf259317bdde4799
I20260807 22:45:35.740264 939 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb0db2be750491daf259317bdde4799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:48346
I20260807 22:45:35.740579 798 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb0db2be750491daf259317bdde4799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:40434
I20260807 22:45:35.741050 641 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb0db2be750491daf259317bdde4799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:35 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:58968
I20260807 22:45:35.741570 1055 tablet_replica.cc:330] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: stopping tablet replica
I20260807 22:45:35.742116 1055 raft_consensus.cc:2229] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:45:35.742986 1055 raft_consensus.cc:2258] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:35.745756 1055 ts_tablet_manager.cc:1905] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:35.749310 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 567
I20260807 22:45:35.753903 1056 tablet_replica.cc:330] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: stopping tablet replica
I20260807 22:45:35.754586 1056 raft_consensus.cc:2229] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:35.755412 1056 raft_consensus.cc:2258] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:35.758029 1055 ts_tablet_manager.cc:1918] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:35.758379 1055 log.cc:1196] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/5eb0db2be750491daf259317bdde4799
I20260807 22:45:35.758574 1056 ts_tablet_manager.cc:1905] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:35.759477 1055 ts_tablet_manager.cc:1939] T 5eb0db2be750491daf259317bdde4799 P 1dcb68a05986444b953fac9be1da2ea4: Deleting consensus metadata
I20260807 22:45:35.761891 467 catalog_manager.cc:4841] TS 1dcb68a05986444b953fac9be1da2ea4 (127.20.10.131:45435): tablet 5eb0db2be750491daf259317bdde4799 (table test-table [id=fc198df09a21404cb07f0c91ce85ba03]) successfully deleted
I20260807 22:45:35.769682 1056 ts_tablet_manager.cc:1918] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:35.769977 1056 log.cc:1196] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/5eb0db2be750491daf259317bdde4799
I20260807 22:45:35.770707 1056 ts_tablet_manager.cc:1939] T 5eb0db2be750491daf259317bdde4799 P 797283d0cdab4f859c15a703152ad517: Deleting consensus metadata
I20260807 22:45:35.773449 467 catalog_manager.cc:4841] TS 797283d0cdab4f859c15a703152ad517 (127.20.10.130:33845): tablet 5eb0db2be750491daf259317bdde4799 (table test-table [id=fc198df09a21404cb07f0c91ce85ba03]) successfully deleted
I20260807 22:45:35.774886 1057 tablet_replica.cc:330] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: stopping tablet replica
I20260807 22:45:35.775614 1057 raft_consensus.cc:2229] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:35.776682 1057 raft_consensus.cc:2258] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:35.779000 1057 ts_tablet_manager.cc:1905] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:45:35 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:45:35.789589 1057 ts_tablet_manager.cc:1918] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:35.789975 1057 log.cc:1196] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/5eb0db2be750491daf259317bdde4799
I20260807 22:45:35.791177 1057 ts_tablet_manager.cc:1939] T 5eb0db2be750491daf259317bdde4799 P 48a8b21da5cc44bea3a3518f07a48c3a: Deleting consensus metadata
I20260807 22:45:35.794147 466 catalog_manager.cc:4841] TS 48a8b21da5cc44bea3a3518f07a48c3a (127.20.10.129:37615): tablet 5eb0db2be750491daf259317bdde4799 (table test-table [id=fc198df09a21404cb07f0c91ce85ba03]) successfully deleted
I20260807 22:45:35.797185 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.751883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.20.10.129:37615)
0807 22:45:35.752078 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.752084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.767391 (+ 15307us) server_negotiation.cc:407] Connection header received
0807 22:45:35.767767 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.767774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.767875 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.767961 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:35.770521 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.771960 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.774528 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.775302 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.775507 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.784521 (+ 9014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:35.784594 (+ 73us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:35.784625 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:35.784698 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:35.791693 (+ 6995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.793391 (+ 1698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.793401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.793405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.793715 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.794479 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.794486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.794490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.795356 (+ 866us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:35.795924 (+ 568us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.796154 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.796452 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:45:35.798446 708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:45:35.899734 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 567
I20260807 22:45:35.931788 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 721
Aug 07 22:45:35 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:45:35.958616 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.932902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.20.10.130:33845)
0807 22:45:35.933079 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:35.933085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:35.933107 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:45:35.933169 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:35.933174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:35.933252 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:35.933365 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:35.938912 (+ 5547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.940476 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.942605 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:35.943415 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:35.943529 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:35.949966 (+ 6437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:35.949991 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:35.950014 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:35.950068 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:35.954592 (+ 4524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.955920 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.955932 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.955936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.956135 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:35.956649 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:35.956663 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:35.956667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:35.957201 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:35.957527 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:35.957857 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 22:45:35.958126 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:45:35.959539 860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:45:36.057263 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 721
I20260807 22:45:36.090751 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 872
Aug 07 22:45:36 dist-test-slave-csg5 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:45:36.111958 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:36.091853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.20.10.131:45435)
0807 22:45:36.092037 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:36.092042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:36.092085 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:45:36.092156 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:36.092160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:36.092226 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:36.092313 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:36.094223 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:36.095308 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:36.097366 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:36.097928 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:36.098064 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:36.104356 (+ 6292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:36.104372 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:36.104390 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:36.104442 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:36.108349 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:36.109355 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:36.109363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:36.109365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:36.109550 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:36.109994 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:36.109999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:36.110007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:36.110514 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:36.110944 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:36.111285 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 22:45:36.111564 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:45:36.112790 1003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:45:36.245798 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 872
I20260807 22:45:36.274564 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 447
I20260807 22:45:36.276150 513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:45:36.403424 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 447
I20260807 22:45:36.440496 20522 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:45:36.787642 20522 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:45:36.787964 20522 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:45:37.145458 20522 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:45:37.145 UTC [32465] LOG: received smart shutdown request
2026-08-07 22:45:37.153 UTC [32465] LOG: background worker "logical replication launcher" (PID 32474) exited with exit code 1
2026-08-07 22:45:37.153 UTC [32469] LOG: shutting down
2026-08-07 22:45:37.383 UTC [32465] LOG: database system is shut down
2026-08-07T22:45:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (61060 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:45:37 dist-test-slave-csg5 krb5kdc[1072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:45:37 dist-test-slave-csg5 krb5kdc[1072](info): set up 2 sockets
Aug 07 22:45:37 dist-test-slave-csg5 krb5kdc[1072](info): commencing operation
krb5kdc: starting...
W20260807 22:45:38.787329 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:45:39 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:45:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangeradmin_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangeradmin_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangerlookup_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangerlookup_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
I20260807 22:45:39.288013 20522 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKr6fn6/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/postgres -l logfile start
2026-08-07 22:45:42.869 UTC [1105] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:45:42.869 UTC [1105] LOG: listening on IPv4 address "127.20.10.148", port 33959
2026-08-07 22:45:42.874 UTC [1105] LOG: listening on Unix socket "/tmp/.s.PGSQL.33959"
2026-08-07 22:45:42.903 UTC [1107] LOG: database system was shut down at 2026-08-07 22:45:40 UTC
2026-08-07 22:45:42.908 UTC [1105] LOG: database system is ready to accept connections
I20260807 22:45:44.034736 20522 mini_postgres.cc:96] Postgres bound to 33959
2026-08-07 22:45:44.064 UTC [1117] FATAL: database "slave" does not exist
127.20.10.148:33959 - accepting connections
I20260807 22:45:44.065881 20522 mini_ranger.cc:161] Starting Ranger...
I20260807 22:45:44.106564 20522 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:45:44.859522 20522 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:45:44.859654 20522 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin
2026-08-07 22:45:45,202 [I] DB FLAVOR :POSTGRES
2026-08-07 22:45:45,203 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:45:45,203 [I] Checking connection..
2026-08-07 22:45:45,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:45:45,516 [I] Checking connection passed.
2026-08-07 22:45:45,516 [I] --------- Verifying version history table ---------
2026-08-07 22:45:45,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:45:45,826 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:45:45,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:45:46,137 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:45:46,137 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:45:46,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:45:46.415 UTC [1187] WARNING: there is no transaction in progress
2026-08-07 22:45:46,425 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:45:46,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:45:46,703 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:45:46,704 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:45:46,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:45:46,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:45:47,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-csg5.c.gcp-upstream.internal','N') ;"
2026-08-07 22:45:47,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:45:47,864 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:45:47,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:45:48,135 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:45:48,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:45:48,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:45:48,684 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:45:48,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:45:49.951 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:49.975 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:49.998 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.019 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.041 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.063 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.139 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.149 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.169 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.184 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.198 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.217 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.230 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.239 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.251 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:50.261 UTC [1305] WARNING: there is no transaction in progress
2026-08-07 22:45:51,256 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:45:51,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:45:51,547 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:45:51,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:45:51,845 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:45:51,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:45:52,150 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:45:52,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:45:52,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:45:52,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-csg5.c.gcp-upstream.internal';"
2026-08-07 22:45:53,065 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:45:53,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:45:53,356 [I] DB_PATCHES have already been applied
I20260807 22:45:53.363499 20522 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKr6fn6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:45:53.363615 20522 mini_ranger.cc:193] Using host: 127.20.10.148
I20260807 22:45:53.389796 20522 mini_ranger.cc:227] Ranger admin URL: http://127.20.10.148:35123
Aug 07, 2026 10:45:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:45:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:45:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:45:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:45:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:45:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:45:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:45:54.675889 20522 mini_ranger.cc:160] Time spent starting Ranger: real 10.610s user 0.003s sys 0.077s
Aug 07, 2026 10:46:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:46:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 10:46:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:46:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:46:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:46:20.558305 20522 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.10.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.10.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangerkms_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.10.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/rangerkms_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.10.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.10.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.10.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/HTTP_127.20.10.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:46:20 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:46:20.812224 20522 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:46:20.845506 20522 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:46:22.004267 20522 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:46:22.004446 20522 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms
I20260807 22:46:22.004545 20522 mini_ranger_kms.cc:227] Using postgres at 127.20.10.148:33959
2026-08-07 22:46:22,434 [I] DB FLAVOR :POSTGRES
2026-08-07 22:46:22,434 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:46:22,434 [I] Checking connection
2026-08-07 22:46:22,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:46:22,714 [I] connection success
2026-08-07 22:46:22,715 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:46:22,715 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:46:22,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:46:22,968 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:46:22,968 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:46:22,969 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:46:22,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.10.148:33959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:46:23,267 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:46:23.870590 20522 mini_ranger_kms.cc:310] Created kms service
I20260807 22:46:24.102350 20522 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:46:24.159184 20522 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:46:24 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:24.647029 20522 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:46:24.647410 20522 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKr6fn6/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKr6fn6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:46:24.647460 20522 mini_ranger_kms.cc:240] Using host: 127.20.10.148
I20260807 22:46:24.677073 20522 mini_ranger_kms.cc:276] Ranger KMS PID: 1501
I20260807 22:46:24.677335 20522 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.10.148:37757
Aug 07, 2026 10:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:46:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:46:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 22:46:27 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:27.188997 20522 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.377s user 0.079s sys 0.392s
I20260807 22:46:27.189251 20522 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:46:27.189438 20522 mini_ranger_kms.cc:401] 127.20.10.148:37757/kms/v1/keys
Aug 07 22:46:27 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
Aug 07 22:46:27 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142787, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.10.148@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
Aug 07 22:46:30 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:46:30.555968 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36517
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:36517
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:30.952452 1554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:30.952953 1554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:30.953130 1554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:30.962716 1554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:30.962929 1554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:30.963037 1554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:30.963173 1554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:30.963277 1554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:30.975869 1554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.10.148:37757/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:36517
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:30.977938 1554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:30.979756 1554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:30.998575 1563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:30.998550 1561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:30.999652 1554 server_base.cc:1034] running on GCE node
W20260807 22:46:30.999831 1560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:31.001117 1554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:31.003568 1554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:31.004873 1554 hybrid_clock.cc:648] HybridClock initialized: now 1786142791004824 us; error 45 us; skew 500 ppm
I20260807 22:46:31.005362 1554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:31 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142791, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:31.013011 1554 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:31.015210 1554 webserver.cc:460] Webserver started at http://127.20.10.190:35557/ using document root <none> and password file <none>
I20260807 22:46:31.015997 1554 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:31.016161 1554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:31.016515 1554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:46:31 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142791, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:31.081916 1554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "13fc4644c9884d71a081fd41da23397b"
format_stamp: "Formatted at 2026-08-07 22:46:31 on dist-test-slave-csg5"
server_key: "bcc4748c0f490832db39d38b1730baa3"
server_key_iv: "a5a437535779c68be84afea14274d7b1"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:31.083171 1554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "13fc4644c9884d71a081fd41da23397b"
format_stamp: "Formatted at 2026-08-07 22:46:31 on dist-test-slave-csg5"
server_key: "bcc4748c0f490832db39d38b1730baa3"
server_key_iv: "a5a437535779c68be84afea14274d7b1"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:31.090140 1554 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:46:31.113026 1570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:31.115018 1554 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:46:31.115275 1554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "13fc4644c9884d71a081fd41da23397b"
format_stamp: "Formatted at 2026-08-07 22:46:31 on dist-test-slave-csg5"
server_key: "bcc4748c0f490832db39d38b1730baa3"
server_key_iv: "a5a437535779c68be84afea14274d7b1"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:31.115561 1554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:31.141808 1554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:31.147018 1554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:31.147449 1554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:31.171211 1554 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:36517
I20260807 22:46:31.171212 1624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:36517 every 8 connection(s)
I20260807 22:46:31.172942 1554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:31.175503 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 1554
I20260807 22:46:31.175774 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:46:31.176546 20522 external_mini_cluster.cc:1428] Setting key 96ee5ea625632218f113f9a13d1a9089
I20260807 22:46:31.179243 1625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:46:31 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:31.195863 1625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Bootstrap starting.
I20260807 22:46:31.201021 1625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:31.202564 1625 log.cc:824] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:31.206884 1625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: No bootstrap required, opened a new log
I20260807 22:46:31.207548 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:31.178805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.20.10.190:36517)
0807 22:46:31.179848 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:31.179865 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:31.179923 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:46:31.182727 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:31.182762 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:31.183381 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:31.183938 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:31.185830 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:31.188293 (+ 2463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:31.190134 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:31.191385 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:31.191763 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:31.196742 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:31.196756 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:31.196780 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:31.196872 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:31.201766 (+ 4894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:31.203000 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:31.203009 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:31.203018 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:31.203252 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:31.203725 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:31.203732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:31.203736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:31.204708 (+ 972us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:31.205104 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:31.205678 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 22:46:31.206184 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":496,"threads_started":1}
I20260807 22:46:31.219136 1625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:31.219956 1625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:31.220145 1625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fc4644c9884d71a081fd41da23397b, State: Initialized, Role: FOLLOWER
I20260807 22:46:31.221048 1625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:31.221367 1625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:31.221508 1625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:31.221683 1625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:31.225148 1625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:31.225905 1625 leader_election.cc:302] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13fc4644c9884d71a081fd41da23397b; no voters:
I20260807 22:46:31.226735 1625 leader_election.cc:288] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:31.226980 1631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:31.229321 1625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:31.231083 1631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 LEADER]: Becoming Leader. State: Replica: 13fc4644c9884d71a081fd41da23397b, State: Running, Role: LEADER
I20260807 22:46:31.231844 1631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:31.236765 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13fc4644c9884d71a081fd41da23397b. Latest consensus state: current_term: 1 leader_uuid: "13fc4644c9884d71a081fd41da23397b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } } }
I20260807 22:46:31.237258 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:31.237854 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13fc4644c9884d71a081fd41da23397b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } } }
I20260807 22:46:31.238204 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:31.251296 1640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:31.260157 1640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:31.264061 1625 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:46:31.297330 1640 catalog_manager.cc:1347] Generated new cluster ID: 0dd7eb7a654d436abe47130f4c36f474
I20260807 22:46:31.297609 1640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:31.333103 1640 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:31.335367 1640 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:31.357090 1640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Generated new TSK 0
I20260807 22:46:31.358458 1640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:32.158572 1670 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:46:32.289299 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:32 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:46:32.802006 1673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:32.802569 1673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:32.802803 1673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:32.813505 1673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:32.813648 1673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:32.813848 1673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:46:32.824920 1673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:32.826476 1673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:32.827983 1673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:32.846947 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:32.855954 1684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:32.856339 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:32.856307 1673 server_base.cc:1034] running on GCE node
I20260807 22:46:32.857446 1673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:32.858953 1673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:32.860318 1673 hybrid_clock.cc:648] HybridClock initialized: now 1786142792860228 us; error 85 us; skew 500 ppm
I20260807 22:46:32.860756 1673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:32 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:32.865077 1673 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:46:32.875098 1673 webserver.cc:460] Webserver started at http://127.20.10.129:35717/ using document root <none> and password file <none>
I20260807 22:46:32.876205 1673 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:32.876381 1673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:32.876825 1673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:46:32 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:32.903779 1673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "95cfec1521b0443988bc7c8aab7c5f1f"
format_stamp: "Formatted at 2026-08-07 22:46:32 on dist-test-slave-csg5"
server_key: "714dae790e8a2bae84b1ed081d65e5d9"
server_key_iv: "cc0acc5421ed135e01cdf3d715a3ed30"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:32.907397 1673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "95cfec1521b0443988bc7c8aab7c5f1f"
format_stamp: "Formatted at 2026-08-07 22:46:32 on dist-test-slave-csg5"
server_key: "714dae790e8a2bae84b1ed081d65e5d9"
server_key_iv: "cc0acc5421ed135e01cdf3d715a3ed30"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:32.920492 1673 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.005s
I20260807 22:46:32.953559 1694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:32.956156 1673 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.004s
I20260807 22:46:32.956535 1673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "95cfec1521b0443988bc7c8aab7c5f1f"
format_stamp: "Formatted at 2026-08-07 22:46:32 on dist-test-slave-csg5"
server_key: "714dae790e8a2bae84b1ed081d65e5d9"
server_key_iv: "cc0acc5421ed135e01cdf3d715a3ed30"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:32.956888 1673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:32.995132 1673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:33.000316 1673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:33.000789 1673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:33.002127 1673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:33.004020 1673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:33.004143 1673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:33.004341 1673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:33.004429 1673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:46:33 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:33.156296 1673 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:42633
I20260807 22:46:33.156347 1817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:42633 every 8 connection(s)
I20260807 22:46:33.158259 1673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:33.164713 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 1673
I20260807 22:46:33.164964 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:33.165921 20522 external_mini_cluster.cc:1428] Setting key 5b67845324a00184ae9bc722374fcff3
Aug 07 22:46:33 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
I20260807 22:46:33.229074 1821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:33.162629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.129:33349)
0807 22:46:33.188895 (+ 26266us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:33.188961 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:33.192391 (+ 3430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:33.202490 (+ 10099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:33.202513 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:33.203448 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:33.204595 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:33.204644 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:33.207027 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:33.207037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:33.209834 (+ 2797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:33.209879 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:33.211516 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:33.211530 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:33.211954 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:33.212988 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:33.213034 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:33.216423 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:33.223134 (+ 6711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:33.223147 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:33.223173 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:33.223973 (+ 800us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:33.224591 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:33.224597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:33.224601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:33.224931 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:33.225816 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:33.225827 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:33.226396 (+ 569us) client_negotiation.cc:769] Sending connection context
0807 22:46:33.226783 (+ 387us) client_negotiation.cc:240] Negotiation successful
0807 22:46:33.227465 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24369,"thread_start_us":24593,"threads_started":1}
I20260807 22:46:33.230388 1820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:33.161827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:33349 (local address 127.20.10.190:36517)
0807 22:46:33.201582 (+ 39755us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:33.201592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:33.201625 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:46:33.201709 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:33.201718 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:33.201846 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:33.204928 (+ 3082us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:33.205194 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:33.206743 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:33.210198 (+ 3455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:33.211147 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:33.214837 (+ 3690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:33.216884 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:33.216895 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:33.216903 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:33.216963 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:33.222830 (+ 5867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:33.224201 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:33.224209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:33.224213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:33.224424 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:33.225121 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:33.225127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:33.225131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:33.225554 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:33.225712 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:33.229576 (+ 3864us) server_negotiation.cc:299] Negotiation successful
0807 22:46:33.229912 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39556,"thread_start_us":39383,"threads_started":1}
I20260807 22:46:33.232903 1818 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:33.233348 1818 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:33.234453 1818 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:33.237917 1587 ts_manager.cc:194] Registered new tserver with Master: 95cfec1521b0443988bc7c8aab7c5f1f (127.20.10.129:42633)
I20260807 22:46:33.240294 1587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:33349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:46:33.266515 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:33 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
W20260807 22:46:33.855260 1824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:33.855798 1824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:33.856081 1824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:33.870028 1824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:33.870405 1824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:33.870767 1824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:46:33.887437 1824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:33.889477 1824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:33.891399 1824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:33.913899 1840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:33.913954 1841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:33.914227 1843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:33.916543 1824 server_base.cc:1034] running on GCE node
I20260807 22:46:33.917358 1824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:33.918587 1824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:33.919893 1824 hybrid_clock.cc:648] HybridClock initialized: now 1786142793919821 us; error 63 us; skew 500 ppm
I20260807 22:46:33.920445 1824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:33 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:33.924731 1824 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:46:33.926872 1824 webserver.cc:460] Webserver started at http://127.20.10.130:38791/ using document root <none> and password file <none>
I20260807 22:46:33.927692 1824 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:33.927846 1824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:33.928215 1824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:46:33 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:33.952610 1824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "75bcc42b4ae541aca927d680420d13f3"
format_stamp: "Formatted at 2026-08-07 22:46:33 on dist-test-slave-csg5"
server_key: "0f43c9f66b1745048b4ae6665050786f"
server_key_iv: "784850431e34e0bcd7b9cab44d10ff23"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:33.954051 1824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "75bcc42b4ae541aca927d680420d13f3"
format_stamp: "Formatted at 2026-08-07 22:46:33 on dist-test-slave-csg5"
server_key: "0f43c9f66b1745048b4ae6665050786f"
server_key_iv: "784850431e34e0bcd7b9cab44d10ff23"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:33.962812 1824 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260807 22:46:33.983405 1850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:33.985610 1824 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:46:33.985877 1824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "75bcc42b4ae541aca927d680420d13f3"
format_stamp: "Formatted at 2026-08-07 22:46:33 on dist-test-slave-csg5"
server_key: "0f43c9f66b1745048b4ae6665050786f"
server_key_iv: "784850431e34e0bcd7b9cab44d10ff23"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:33.986203 1824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:34.006582 1824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:34.011449 1824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:34.011801 1824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:34.013036 1824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:34.014770 1824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:34.014858 1824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:34.015007 1824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:34.015066 1824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:34.057729 1824 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:42143
I20260807 22:46:34.057741 1970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:42143 every 8 connection(s)
I20260807 22:46:34.059247 1824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:34.065734 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 1824
I20260807 22:46:34.065933 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:34.066540 20522 external_mini_cluster.cc:1428] Setting key 2569e3dc413d6f2ea160cc4c7a7a5245
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:34.087525 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.062507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:42747 (local address 127.20.10.190:36517)
0807 22:46:34.063060 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:34.063066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:34.064292 (+ 1226us) server_negotiation.cc:407] Connection header received
0807 22:46:34.067252 (+ 2960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:34.067258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:34.067356 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:34.067488 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:34.069518 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.070299 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.072910 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.073573 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.073730 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.079015 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:34.079023 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:34.079029 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:34.079088 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:34.083024 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.084292 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.084299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.084302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.084483 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.085185 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.085190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.085192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.085662 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:34.085905 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:34.086788 (+ 883us) server_negotiation.cc:299] Negotiation successful
0807 22:46:34.087073 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":285,"threads_started":1}
I20260807 22:46:34.088649 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.063213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.130:42747)
0807 22:46:34.064098 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:34.064147 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:34.066645 (+ 2498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:34.067810 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:34.067820 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:34.068522 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:34.069323 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:34.069346 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.070477 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.070483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:34.072699 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:34.072722 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.073882 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.073892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:34.074276 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.075348 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:34.075393 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:34.078569 (+ 3176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:34.083256 (+ 4687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:34.083270 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:34.083295 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:34.084098 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:34.084632 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:34.084640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:34.084647 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:34.085007 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:34.085859 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:34.085869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:34.086304 (+ 435us) client_negotiation.cc:769] Sending connection context
0807 22:46:34.086621 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 22:46:34.087281 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":417,"threads_started":1}
I20260807 22:46:34.090358 1971 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:34.090807 1971 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:34.091756 1971 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:34.093581 1587 ts_manager.cc:194] Registered new tserver with Master: 75bcc42b4ae541aca927d680420d13f3 (127.20.10.130:42143)
I20260807 22:46:34.094569 1587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:42747
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:46:34.156998 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:34.245294 1818 heartbeater.cc:503] Master 127.20.10.190:36517 was elected leader, sending a full tablet report...
W20260807 22:46:34.550753 1978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:34.551075 1978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:34.551174 1978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:34.558938 1978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:34.559074 1978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:34.559192 1978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:46:34.569913 1978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:34.571594 1978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:34.572945 1978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:34.588294 1986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:34.588470 1984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:34.588937 1983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:34.589361 1978 server_base.cc:1034] running on GCE node
I20260807 22:46:34.590111 1978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:34.591311 1978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:34.592628 1978 hybrid_clock.cc:648] HybridClock initialized: now 1786142794592548 us; error 61 us; skew 500 ppm
I20260807 22:46:34.593037 1978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:34.596777 1978 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:46:34.598801 1978 webserver.cc:460] Webserver started at http://127.20.10.131:36497/ using document root <none> and password file <none>
I20260807 22:46:34.599627 1978 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:34.599798 1978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:34.600169 1978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:34.617425 1978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "fc9741af9b9b4897ab069fd7f32e6553"
format_stamp: "Formatted at 2026-08-07 22:46:34 on dist-test-slave-csg5"
server_key: "ff38349233ad31c380589a8be209047b"
server_key_iv: "87b888eee692f8f79c686a66f24f2b14"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:34.618381 1978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "fc9741af9b9b4897ab069fd7f32e6553"
format_stamp: "Formatted at 2026-08-07 22:46:34 on dist-test-slave-csg5"
server_key: "ff38349233ad31c380589a8be209047b"
server_key_iv: "87b888eee692f8f79c686a66f24f2b14"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:34.625742 1978 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260807 22:46:34.645934 1993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:34.647851 1978 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:46:34.648039 1978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "fc9741af9b9b4897ab069fd7f32e6553"
format_stamp: "Formatted at 2026-08-07 22:46:34 on dist-test-slave-csg5"
server_key: "ff38349233ad31c380589a8be209047b"
server_key_iv: "87b888eee692f8f79c686a66f24f2b14"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:34.648308 1978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:34.680467 1978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:34.686916 1978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:34.687301 1978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:34.688687 1978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:34.691090 1978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:34.691216 1978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:34.691411 1978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:34.691488 1978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:34.738755 1978 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43849
I20260807 22:46:34.738803 2119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43849 every 8 connection(s)
I20260807 22:46:34.740686 1978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:34.743620 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 1978
I20260807 22:46:34.743832 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:34.744380 20522 external_mini_cluster.cc:1428] Setting key d5121eb819871be9aa72b0a1c8232e51
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:34.770375 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.745282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:53359 (local address 127.20.10.190:36517)
0807 22:46:34.749630 (+ 4348us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:34.749638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:34.750111 (+ 473us) server_negotiation.cc:407] Connection header received
0807 22:46:34.752393 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:34.752399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:34.752518 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:34.752663 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:34.754488 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.755436 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.757229 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.757730 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.757887 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.762552 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:34.762559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:34.762565 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:34.762619 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:34.766282 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.767496 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.767501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.767504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.767671 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.768310 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.768315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.768317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.768654 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:34.768786 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:34.769832 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0807 22:46:34.770043 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4190,"thread_start_us":3988,"threads_started":1}
I20260807 22:46:34.771895 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.744533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.131:53359)
0807 22:46:34.749937 (+ 5404us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:34.749986 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:34.752072 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:34.752961 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:34.752970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:34.753521 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:34.754288 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:34.754310 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.755617 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.755622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:34.757058 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:34.757072 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.758028 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.758036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:34.758308 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.759149 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:34.759220 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:34.762277 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:34.766512 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:34.766524 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:34.766545 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:34.767336 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:34.767811 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:34.767816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:34.767820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:34.768141 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:34.768807 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:34.768817 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:34.769324 (+ 507us) client_negotiation.cc:769] Sending connection context
0807 22:46:34.769681 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 22:46:34.770287 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5100,"thread_start_us":4835,"threads_started":1}
I20260807 22:46:34.774293 2120 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:34.774770 2120 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:34.775771 2120 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:34.777644 1586 ts_manager.cc:194] Registered new tserver with Master: fc9741af9b9b4897ab069fd7f32e6553 (127.20.10.131:43849)
I20260807 22:46:34.778832 1586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:53359
I20260807 22:46:34.788414 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:34.790405 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 1673
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:46:34.815143 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.792018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.20.10.129:42633)
0807 22:46:34.793152 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:34.793160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:34.793190 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:46:34.793265 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:34.793272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:34.793524 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:34.793675 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:34.795987 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.797344 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.799671 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.800426 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.800553 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.806713 (+ 6160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:34.806739 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:34.806763 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:34.806823 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:34.811453 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.812424 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.812436 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.812440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.812673 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.813139 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.813145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.813149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.813677 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:34.814038 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:34.814289 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:46:34.814603 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":374,"threads_started":1}
I20260807 22:46:34.816908 1813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:34.911121 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 1673
I20260807 22:46:34.936110 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 1824
Aug 07 22:46:34 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:46:34.958923 2127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.937190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.20.10.130:42143)
0807 22:46:34.937853 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:34.937865 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:34.937924 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:46:34.938016 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:34.938027 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:34.938373 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:34.938568 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:34.940351 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.941860 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.944088 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:34.944990 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:34.945172 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:34.951143 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:34.951167 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:34.951189 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:34.951245 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:34.955301 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.956303 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.956314 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.956317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.956498 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:34.956959 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:34.956964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:34.956967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:34.957435 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:34.957762 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:34.958152 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 22:46:34.958439 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":339,"threads_started":1}
I20260807 22:46:34.960547 1966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:35.050383 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 1824
I20260807 22:46:35.079926 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 1978
Aug 07 22:46:35 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:46:35.099525 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:35.081315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.20.10.131:43849)
0807 22:46:35.082080 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:35.082087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:35.082117 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:46:35.082186 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:35.082193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:35.082416 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:35.082606 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:35.084040 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:35.085074 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:35.086470 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:35.087156 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:35.087312 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:35.092695 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:35.092711 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:35.092727 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:35.092770 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:35.096189 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:35.097002 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:35.097010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:35.097012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:35.097151 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:35.097553 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:35.097558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:35.097561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:35.097941 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:35.098312 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:35.098607 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 22:46:35.099090 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":426,"threads_started":1}
I20260807 22:46:35.101069 2113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:35.205662 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 1978
I20260807 22:46:35.226429 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 1554
I20260807 22:46:35.228014 1620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:35.343534 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 1554
I20260807 22:46:35.372541 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36517
--webserver_interface=127.20.10.190
--webserver_port=35557
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:36517
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:35.746420 2133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:35.746816 2133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:35.746923 2133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:35.755472 2133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:35.755612 2133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:35.755666 2133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:35.755720 2133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:35.755775 2133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:35.767935 2133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:36517
--ranger_config_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=35557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:35.769419 2133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:35.770900 2133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:35.785508 2138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:35.785508 2141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:35.785547 2139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:35.786592 2133 server_base.cc:1034] running on GCE node
I20260807 22:46:35.787886 2133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:35.789671 2133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:35.790897 2133 hybrid_clock.cc:648] HybridClock initialized: now 1786142795790859 us; error 37 us; skew 500 ppm
I20260807 22:46:35.791477 2133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:35 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:35.795456 2133 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:35.797168 2133 webserver.cc:460] Webserver started at http://127.20.10.190:35557/ using document root <none> and password file <none>
I20260807 22:46:35.797901 2133 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:35.798045 2133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:46:35 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:35.847633 2133 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 22:46:35.851562 2149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:35.853201 2133 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:35.853458 2133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "13fc4644c9884d71a081fd41da23397b"
format_stamp: "Formatted at 2026-08-07 22:46:31 on dist-test-slave-csg5"
server_key: "bcc4748c0f490832db39d38b1730baa3"
server_key_iv: "a5a437535779c68be84afea14274d7b1"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:35.854658 2133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:35.892190 2133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:35.897225 2133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:35.897581 2133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:35.918905 2133 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:36517
I20260807 22:46:35.918924 2206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:36517 every 8 connection(s)
I20260807 22:46:35.920655 2133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:35.928112 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2133
I20260807 22:46:35.928243 2207 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:46:35.929121 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:42633
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:36517
--webserver_port=35717
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:35.932010 2207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Bootstrap starting.
I20260807 22:46:35.956239 2207 log.cc:824] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:35.969790 2207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:46:35.970535 2207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Bootstrap complete.
I20260807 22:46:35.985615 2207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:35.987474 2207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fc4644c9884d71a081fd41da23397b, State: Initialized, Role: FOLLOWER
I20260807 22:46:35.988452 2207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:35.988790 2207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:35.988934 2207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:35.989159 2207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:46:35.995992 2207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:35.996798 2207 leader_election.cc:302] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13fc4644c9884d71a081fd41da23397b; no voters:
I20260807 22:46:35.997689 2207 leader_election.cc:288] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:46:35.997876 2211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:46:35.999377 2211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [term 2 LEADER]: Becoming Leader. State: Replica: 13fc4644c9884d71a081fd41da23397b, State: Running, Role: LEADER
I20260807 22:46:36.000056 2211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } }
I20260807 22:46:36.000463 2207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:36.003049 2212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "13fc4644c9884d71a081fd41da23397b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } } }
I20260807 22:46:36.003494 2212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:36.004096 2213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13fc4644c9884d71a081fd41da23397b. Latest consensus state: current_term: 2 leader_uuid: "13fc4644c9884d71a081fd41da23397b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc4644c9884d71a081fd41da23397b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36517 } } }
I20260807 22:46:36.004458 2213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:36.015344 2217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:36.023171 2217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:36.024446 2217 catalog_manager.cc:1259] Loaded cluster ID: 0dd7eb7a654d436abe47130f4c36f474
I20260807 22:46:36.024600 2217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:36.028967 2217 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:36.030929 2217 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Loaded TSK: 0
I20260807 22:46:36.032907 2217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:36.036893 2207 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 22:46:36.488705 2209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:36.489310 2209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:36.489557 2209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:36.502569 2209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:36.502789 2209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:36.503043 2209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:46:36.515625 2209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:42633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=35717
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:36.517426 2209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:36.519193 2209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:36.539760 2247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:36.540512 2248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:36.549820 2209 server_base.cc:1034] running on GCE node
W20260807 22:46:36.550305 2251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:36.551397 2209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:36.555085 2209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:36.556501 2209 hybrid_clock.cc:648] HybridClock initialized: now 1786142796556444 us; error 47 us; skew 500 ppm
I20260807 22:46:36.557049 2209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:36 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142796, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:36.571511 2209 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:46:36.575206 2209 webserver.cc:460] Webserver started at http://127.20.10.129:35717/ using document root <none> and password file <none>
I20260807 22:46:36.576445 2209 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:36.576669 2209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:46:36 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142796, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:36.616150 2209 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.006s sys 0.001s
I20260807 22:46:36.623979 2260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:36.626082 2209 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.008s sys 0.000s
I20260807 22:46:36.626333 2209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "95cfec1521b0443988bc7c8aab7c5f1f"
format_stamp: "Formatted at 2026-08-07 22:46:32 on dist-test-slave-csg5"
server_key: "714dae790e8a2bae84b1ed081d65e5d9"
server_key_iv: "cc0acc5421ed135e01cdf3d715a3ed30"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:36.627630 2209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:36.646458 2209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:36.652292 2209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:36.652678 2209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:36.653954 2209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:36.655915 2209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:36.656055 2209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:36.656255 2209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:36.656352 2209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:36.725217 2209 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:42633
I20260807 22:46:36.725289 2383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:42633 every 8 connection(s)
I20260807 22:46:36.728083 2209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:36.738481 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2209
I20260807 22:46:36.739378 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:42143
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:36517
--webserver_port=38791
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:36 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142796, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:36.790793 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:36.733200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.129:32977)
0807 22:46:36.753197 (+ 19997us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:36.753271 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:36.756225 (+ 2954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:36.764144 (+ 7919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:36.764162 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:36.765062 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:36.766172 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:36.766208 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:36.769243 (+ 3035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:36.769250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:36.771560 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:36.771583 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:36.773693 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:36.773705 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:36.774018 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:36.775346 (+ 1328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:36.775380 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:36.778153 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:36.784379 (+ 6226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:36.784392 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:36.784413 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:36.785203 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:36.785923 (+ 720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:36.785929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:36.785934 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:36.786285 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:36.787655 (+ 1370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:36.787665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:36.788147 (+ 482us) client_negotiation.cc:769] Sending connection context
0807 22:46:36.788485 (+ 338us) client_negotiation.cc:240] Negotiation successful
0807 22:46:36.789154 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19630,"thread_start_us":19359,"threads_started":1}
I20260807 22:46:36.793780 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:36.734807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:32977 (local address 127.20.10.190:36517)
0807 22:46:36.752449 (+ 17642us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:36.752474 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:36.756831 (+ 4357us) server_negotiation.cc:407] Connection header received
0807 22:46:36.757393 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:36.757418 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:36.758061 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:36.758527 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:36.766535 (+ 8008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:36.768931 (+ 2396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:36.771851 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:36.773018 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:36.773244 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:36.778599 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:36.778617 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:36.778644 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:36.778738 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:36.784056 (+ 5318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:36.785458 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:36.785470 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:36.785480 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:36.785745 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:36.786465 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:36.786471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:36.786476 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:36.787289 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:36.787559 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:36.791665 (+ 4106us) server_negotiation.cc:299] Negotiation successful
0807 22:46:36.792198 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17297,"thread_start_us":17046,"threads_started":1}
I20260807 22:46:36.797902 2384 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:36.798290 2384 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:36.799288 2384 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
W20260807 22:46:36.803241 2384 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:36517 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:46:36.803520 2384 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:36.803889 2384 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:36.805469 2384 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:36.805894 2384 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
W20260807 22:46:36.807318 2384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:46:37.218355 2388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.218770 2388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.218986 2388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.228421 2388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.228566 2388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.228775 2388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:46:37.240980 2388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:42143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=38791
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:37.242870 2388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.244552 2388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:37.265475 2404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.266940 2405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.273568 2388 server_base.cc:1034] running on GCE node
W20260807 22:46:37.273878 2407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.274901 2388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:37.276877 2388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:37.278183 2388 hybrid_clock.cc:648] HybridClock initialized: now 1786142797278102 us; error 75 us; skew 500 ppm
I20260807 22:46:37.278817 2388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:37 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:37.283102 2388 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:46:37.285449 2388 webserver.cc:460] Webserver started at http://127.20.10.130:38791/ using document root <none> and password file <none>
I20260807 22:46:37.286383 2388 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:37.286573 2388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:46:37 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:37.316138 2388 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 22:46:37.332086 2416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.334482 2388 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.010s sys 0.004s
I20260807 22:46:37.334754 2388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "75bcc42b4ae541aca927d680420d13f3"
format_stamp: "Formatted at 2026-08-07 22:46:33 on dist-test-slave-csg5"
server_key: "0f43c9f66b1745048b4ae6665050786f"
server_key_iv: "784850431e34e0bcd7b9cab44d10ff23"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:37.336342 2388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:37.391088 2388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:37.396661 2388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:37.397017 2388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:37.398473 2388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:37.401619 2388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:37.401764 2388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.401957 2388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:37.402064 2388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.460952 2388 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:42143
I20260807 22:46:37.461006 2543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:42143 every 8 connection(s)
I20260807 22:46:37.463052 2388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:37.471120 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2388
I20260807 22:46:37.472136 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.10.148:37757/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:43849
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:36517
--webserver_port=36497
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:37 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 22:46:37 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:37.499696 2547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.468780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:51595 (local address 127.20.10.190:36517)
0807 22:46:37.472466 (+ 3686us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.472475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.472507 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:46:37.476185 (+ 3678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.476196 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.476334 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.476455 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:37.479207 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.480640 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.483892 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.484781 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.484928 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.490571 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:37.490581 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:37.490589 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:37.490642 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:37.494870 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.496057 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.496066 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.496072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.496248 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.496892 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.496900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.496905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.497338 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:37.497492 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:37.498864 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0807 22:46:37.499174 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3489,"thread_start_us":6787,"threads_started":1}
I20260807 22:46:37.502503 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.467860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.130:51595)
0807 22:46:37.470478 (+ 2618us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:37.470554 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:37.475650 (+ 5096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:37.477010 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:37.477027 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:37.477831 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:37.478916 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.478954 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.481014 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.481024 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.483591 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.483616 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.485418 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.485429 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.485851 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.486863 (+ 1012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:37.486906 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:37.490137 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:37.495089 (+ 4952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.495102 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.495126 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.495868 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.496414 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.496422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.496428 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.496735 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.497569 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:37.497581 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:37.498095 (+ 514us) client_negotiation.cc:769] Sending connection context
0807 22:46:37.498450 (+ 355us) client_negotiation.cc:240] Negotiation successful
0807 22:46:37.500853 (+ 2403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2260,"thread_start_us":1951,"threads_started":1}
I20260807 22:46:37.504874 2544 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:37.505255 2544 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:37.506258 2544 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:37.508971 2169 ts_manager.cc:194] Registered new tserver with Master: 75bcc42b4ae541aca927d680420d13f3 (127.20.10.130:42143)
I20260807 22:46:37.511806 2169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:51595
I20260807 22:46:37.807479 2384 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:37.807966 2384 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:37.809217 2169 ts_manager.cc:194] Registered new tserver with Master: 95cfec1521b0443988bc7c8aab7c5f1f (127.20.10.129:42633)
I20260807 22:46:37.810134 2169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:32977
W20260807 22:46:37.943163 2548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.943672 2548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.943871 2548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.957680 2548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.957880 2548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.958117 2548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:46:37.976984 2548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.10.148:37757/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:43849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=36497
--tserver_master_addrs=127.20.10.190:36517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:37.979023 2548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.981033 2548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:38.006304 2557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.011137 2556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.021265 2562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:38.022919 2548 server_base.cc:1034] running on GCE node
I20260807 22:46:38.023919 2548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:38.035104 2548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 07 22:46:38 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:38.042789 2548 hybrid_clock.cc:648] HybridClock initialized: now 1786142798042662 us; error 91 us; skew 500 ppm
I20260807 22:46:38.043416 2548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:38 dist-test-slave-csg5 krb5kdc[1072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142798, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:38.049090 2548 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:46:38.102109 2548 webserver.cc:460] Webserver started at http://127.20.10.131:36497/ using document root <none> and password file <none>
I20260807 22:46:38.103196 2548 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:38.103385 2548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:46:38 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142798, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:38.141304 2548 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.005s sys 0.003s
I20260807 22:46:38.145872 2570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.147881 2548 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 22:46:38.148094 2548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "fc9741af9b9b4897ab069fd7f32e6553"
format_stamp: "Formatted at 2026-08-07 22:46:34 on dist-test-slave-csg5"
server_key: "ff38349233ad31c380589a8be209047b"
server_key_iv: "87b888eee692f8f79c686a66f24f2b14"
server_key_version: "kuduclusterkey@0"
I20260807 22:46:38.149507 2548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:38.175143 2548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:38.182907 2548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:38.183279 2548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:38.184993 2548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:38.187681 2548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:38.187812 2548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.187920 2548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:38.187996 2548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.239068 2548 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43849
I20260807 22:46:38.239173 2692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43849 every 8 connection(s)
I20260807 22:46:38.241147 2548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:38.241911 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2548
I20260807 22:46:38.277796 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.243717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.20.10.190:36517)
0807 22:46:38.252217 (+ 8500us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.252228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.252254 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:46:38.252359 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.252368 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.252495 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.252639 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.255719 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.256948 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.262807 (+ 5859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.263495 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.263630 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.270839 (+ 7209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.270850 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.270855 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:38.270914 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.274631 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.275556 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.275562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.275571 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.275707 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.276136 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.276147 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.276149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.276498 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.276797 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.277117 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.277386 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8324,"thread_start_us":8183,"threads_started":1}
Aug 07 22:46:38 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142798, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:38.295243 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.252097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:39533 (local address 127.20.10.190:36517)
0807 22:46:38.271094 (+ 18997us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.271100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.271125 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:46:38.271851 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.271858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.271979 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.272097 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.274517 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.275868 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.278420 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.279535 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.279667 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.285028 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.285036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.285042 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:38.285113 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.289680 (+ 4567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.290571 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.290579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.290582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.290802 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.291406 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.291412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.291415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.291819 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.292253 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.294440 (+ 2187us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.294810 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18811,"thread_start_us":18601,"threads_started":1}
I20260807 22:46:38.296074 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.251406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:36517 (local address 127.20.10.131:39533)
0807 22:46:38.263397 (+ 11991us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.263463 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.271298 (+ 7835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.272494 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.272509 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.273211 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:38.274170 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.274202 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.276082 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.276088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.278250 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.278268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.280051 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.280060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.280414 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.281589 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:38.281615 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:38.284595 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:38.289885 (+ 5290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:38.289901 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:38.289918 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:38.290421 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:38.290954 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:38.290958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:38.290961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:38.291238 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:38.292383 (+ 1145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:38.292393 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:38.292963 (+ 570us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.293398 (+ 435us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.294096 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11625,"thread_start_us":11366,"threads_started":1}
I20260807 22:46:38.297712 2693 heartbeater.cc:347] Connected to a master server at 127.20.10.190:36517
I20260807 22:46:38.298087 2693 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:38.299024 2693 heartbeater.cc:511] Master 127.20.10.190:36517 requested a full tablet report, sending...
I20260807 22:46:38.300827 2169 ts_manager.cc:194] Registered new tserver with Master: fc9741af9b9b4897ab069fd7f32e6553 (127.20.10.131:43849)
I20260807 22:46:38.301894 2169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:39533
I20260807 22:46:38.306972 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:38.329914 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.310857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.20.10.190:36517)
0807 22:46:38.311297 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.311305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.311648 (+ 343us) server_negotiation.cc:407] Connection header received
0807 22:46:38.311940 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.311947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.312051 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.312173 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.313951 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.314861 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.316863 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.317560 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.317722 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.322500 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.322508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.322513 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:38.322566 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.326343 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.327447 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.327453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.327457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.327628 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.328159 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.328164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.328168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.328548 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.328800 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.329307 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.329579 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260807 22:46:38.337054 2169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:38.358559 2169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 22:46:38 dist-test-slave-csg5 krb5kdc[1072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for HTTP/127.20.10.148@KRBTEST.COM
I20260807 22:46:38.437124 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.383667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:42633 (local address 127.0.0.1:45964)
0807 22:46:38.421882 (+ 38215us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.421909 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.422101 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.423929 (+ 1828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.423938 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.423983 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.424342 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.424354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.427282 (+ 2928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.427288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.430306 (+ 3018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.430327 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.435482 (+ 5155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.435491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.435790 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.435876 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.436355 (+ 479us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.436645 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38055,"thread_start_us":38546,"threads_started":1}
I20260807 22:46:38.438822 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.384651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.20.10.129:42633)
0807 22:46:38.421091 (+ 36440us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.421102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.422079 (+ 977us) server_negotiation.cc:407] Connection header received
0807 22:46:38.422337 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.422350 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.422810 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.423112 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.424584 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.427073 (+ 2489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.432844 (+ 5771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.434639 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.434843 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.435155 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.437608 (+ 2453us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.437869 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36196,"thread_start_us":36456,"threads_started":1}
I20260807 22:46:38.445595 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.379182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43849 (local address 127.0.0.1:38190)
0807 22:46:38.384603 (+ 5421us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.384732 (+ 129us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.385135 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.424820 (+ 39685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.424826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.424858 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.425285 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.425302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.439874 (+ 14572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.439883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.441562 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.441625 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.444575 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.444583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.444830 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.444886 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.444975 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.445166 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5057,"thread_start_us":11119,"threads_started":1}
I20260807 22:46:38.447697 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.387303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.20.10.131:43849)
0807 22:46:38.423974 (+ 36671us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.423989 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.424041 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:46:38.424178 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.424191 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.424548 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.424852 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.430614 (+ 5762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.432560 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.442149 (+ 9589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.444257 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.446374 (+ 2117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.446965 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.447073 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.447282 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36395,"thread_start_us":36177,"threads_started":1}
I20260807 22:46:38.452818 2620 tablet_service.cc:1467] Processing CreateTablet for tablet 93691199a6ac4df5a8e019f92e353083 (DEFAULT_TABLE table=test-table [id=8b62d8d8802249c7a88ecb7082be71d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.454613 2620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93691199a6ac4df5a8e019f92e353083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.456581 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.423131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:42143 (local address 127.0.0.1:43658)
0807 22:46:38.443045 (+ 19914us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.443074 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.446821 (+ 3747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.448786 (+ 1965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.448792 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.448829 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.449048 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.449061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.450992 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.450999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.453136 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.453159 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.455641 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.455648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.455872 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.455929 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.456033 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.456246 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19781,"threads_started":1}
I20260807 22:46:38.457751 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.423307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.20.10.130:42143)
0807 22:46:38.442879 (+ 19572us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.442894 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.443389 (+ 495us) server_negotiation.cc:407] Connection header received
0807 22:46:38.447103 (+ 3714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.447116 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.447471 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.447726 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.449298 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.450756 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.453421 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.455276 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.455429 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.457059 (+ 1630us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.457149 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.457375 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19307,"thread_start_us":18930,"threads_started":1}
I20260807 22:46:38.477123 2471 tablet_service.cc:1467] Processing CreateTablet for tablet 93691199a6ac4df5a8e019f92e353083 (DEFAULT_TABLE table=test-table [id=8b62d8d8802249c7a88ecb7082be71d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.478921 2471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93691199a6ac4df5a8e019f92e353083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.495375 2311 tablet_service.cc:1467] Processing CreateTablet for tablet 93691199a6ac4df5a8e019f92e353083 (DEFAULT_TABLE table=test-table [id=8b62d8d8802249c7a88ecb7082be71d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.497140 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93691199a6ac4df5a8e019f92e353083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.514355 2716 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Bootstrap starting.
I20260807 22:46:38.518236 2544 heartbeater.cc:503] Master 127.20.10.190:36517 was elected leader, sending a full tablet report...
I20260807 22:46:38.519073 2716 tablet_bootstrap.cc:654] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.520716 2716 log.cc:824] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.521677 2719 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Bootstrap starting.
I20260807 22:46:38.525796 2719 tablet_bootstrap.cc:654] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.528242 2718 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Bootstrap starting.
I20260807 22:46:38.529645 2719 log.cc:824] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.533241 2718 tablet_bootstrap.cc:654] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.535732 2718 log.cc:824] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.539211 2716 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: No bootstrap required, opened a new log
I20260807 22:46:38.539738 2716 ts_tablet_manager.cc:1397] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.005s
I20260807 22:46:38.545225 2719 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: No bootstrap required, opened a new log
I20260807 22:46:38.545696 2719 ts_tablet_manager.cc:1397] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.004s
I20260807 22:46:38.548786 2718 tablet_bootstrap.cc:492] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: No bootstrap required, opened a new log
I20260807 22:46:38.549242 2718 ts_tablet_manager.cc:1397] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.008s
I20260807 22:46:38.556157 2716 raft_consensus.cc:348] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.556968 2716 raft_consensus.cc:374] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.557157 2716 raft_consensus.cc:729] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75bcc42b4ae541aca927d680420d13f3, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.557837 2719 raft_consensus.cc:348] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.558368 2719 raft_consensus.cc:374] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.558101 2716 consensus_queue.cc:260] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.558571 2719 raft_consensus.cc:729] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95cfec1521b0443988bc7c8aab7c5f1f, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.559767 2719 consensus_queue.cc:260] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.567173 2716 ts_tablet_manager.cc:1428] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Time spent starting tablet: real 0.027s user 0.013s sys 0.009s
I20260807 22:46:38.568436 2719 ts_tablet_manager.cc:1428] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Time spent starting tablet: real 0.023s user 0.015s sys 0.005s
I20260807 22:46:38.568913 2384 heartbeater.cc:503] Master 127.20.10.190:36517 was elected leader, sending a full tablet report...
I20260807 22:46:38.574903 2718 raft_consensus.cc:348] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.575630 2718 raft_consensus.cc:374] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.575819 2718 raft_consensus.cc:729] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9741af9b9b4897ab069fd7f32e6553, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.576836 2718 consensus_queue.cc:260] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.580508 2693 heartbeater.cc:503] Master 127.20.10.190:36517 was elected leader, sending a full tablet report...
I20260807 22:46:38.584085 2718 ts_tablet_manager.cc:1428] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Time spent starting tablet: real 0.035s user 0.016s sys 0.003s
W20260807 22:46:38.722920 2545 tablet.cc:2369] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:38.734202 2725 raft_consensus.cc:482] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:38.734543 2725 raft_consensus.cc:504] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.736761 2725 leader_election.cc:288] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc9741af9b9b4897ab069fd7f32e6553 (127.20.10.131:43849), 75bcc42b4ae541aca927d680420d13f3 (127.20.10.130:42143)
I20260807 22:46:38.746152 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.737886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:50853 (local address 127.20.10.130:42143)
0807 22:46:38.738093 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.738099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.739025 (+ 926us) server_negotiation.cc:407] Connection header received
0807 22:46:38.739430 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.739438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.739534 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.739652 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.740334 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.741371 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.744110 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.744995 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.745227 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.745316 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.745709 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.745862 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:46:38.746232 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.737342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:42143 (local address 127.20.10.129:50853)
0807 22:46:38.738915 (+ 1573us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.738934 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.739137 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.739676 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.739681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.739709 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.740170 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.740184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.741516 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.741522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.743944 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.743959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.745268 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.745275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.745470 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.745506 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.745664 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.745821 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"thread_start_us":754,"threads_started":1}
I20260807 22:46:38.746299 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.737466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:48757 (local address 127.20.10.131:43849)
0807 22:46:38.737751 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.737762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.738598 (+ 836us) server_negotiation.cc:407] Connection header received
0807 22:46:38.739127 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.739132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.739227 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.739340 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.740035 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.740973 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.743843 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.745066 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.745268 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.745343 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.745914 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.746049 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:46:38.747023 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.737025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43849 (local address 127.20.10.129:48757)
0807 22:46:38.738433 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.738461 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.738771 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.739369 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.739374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.739409 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.739854 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.739867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.741129 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.741136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.743645 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.743662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.745527 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.745534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.745713 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.745747 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.746549 (+ 802us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.746714 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":1503,"threads_started":1}
I20260807 22:46:38.747772 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93691199a6ac4df5a8e019f92e353083" candidate_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75bcc42b4ae541aca927d680420d13f3" is_pre_election: true
I20260807 22:46:38.748181 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93691199a6ac4df5a8e019f92e353083" candidate_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9741af9b9b4897ab069fd7f32e6553" is_pre_election: true
I20260807 22:46:38.748401 2494 raft_consensus.cc:2454] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95cfec1521b0443988bc7c8aab7c5f1f in term 0.
I20260807 22:46:38.748615 2642 raft_consensus.cc:2454] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95cfec1521b0443988bc7c8aab7c5f1f in term 0.
I20260807 22:46:38.749277 2262 leader_election.cc:302] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75bcc42b4ae541aca927d680420d13f3, 95cfec1521b0443988bc7c8aab7c5f1f; no voters:
I20260807 22:46:38.749747 2725 raft_consensus.cc:2781] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:38.749931 2725 raft_consensus.cc:482] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:38.750078 2725 raft_consensus.cc:3037] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.752722 2725 raft_consensus.cc:504] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.754021 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93691199a6ac4df5a8e019f92e353083" candidate_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9741af9b9b4897ab069fd7f32e6553"
I20260807 22:46:38.754164 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93691199a6ac4df5a8e019f92e353083" candidate_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75bcc42b4ae541aca927d680420d13f3"
I20260807 22:46:38.754287 2642 raft_consensus.cc:3037] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.754417 2494 raft_consensus.cc:3037] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:46:38.757182 2696 tablet.cc:2369] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:38.758086 2642 raft_consensus.cc:2454] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95cfec1521b0443988bc7c8aab7c5f1f in term 1.
I20260807 22:46:38.758086 2494 raft_consensus.cc:2454] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95cfec1521b0443988bc7c8aab7c5f1f in term 1.
I20260807 22:46:38.753535 2725 leader_election.cc:288] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [CANDIDATE]: Term 1 election: Requested vote from peers fc9741af9b9b4897ab069fd7f32e6553 (127.20.10.131:43849), 75bcc42b4ae541aca927d680420d13f3 (127.20.10.130:42143)
I20260807 22:46:38.758613 2262 leader_election.cc:302] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75bcc42b4ae541aca927d680420d13f3, 95cfec1521b0443988bc7c8aab7c5f1f; no voters:
I20260807 22:46:38.758953 2725 raft_consensus.cc:2781] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:38.759586 2725 raft_consensus.cc:686] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 1 LEADER]: Becoming Leader. State: Replica: 95cfec1521b0443988bc7c8aab7c5f1f, State: Running, Role: LEADER
I20260807 22:46:38.760428 2725 consensus_queue.cc:237] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } }
I20260807 22:46:38.765995 2169 catalog_manager.cc:5495] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 95cfec1521b0443988bc7c8aab7c5f1f (127.20.10.129). New cstate: current_term: 1 leader_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95cfec1521b0443988bc7c8aab7c5f1f" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 42633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 } health_report { overall_health: UNKNOWN } } }
W20260807 22:46:38.776970 2385 tablet.cc:2369] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:38.858342 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.852096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.20.10.129:42633)
0807 22:46:38.852319 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.852327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.852664 (+ 337us) server_negotiation.cc:407] Connection header received
0807 22:46:38.852942 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.852946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.853053 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.853221 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:38.853652 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.854427 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.856042 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.856459 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.856602 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.857170 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:38.857715 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:38.857881 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.857964 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.858051 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:46:38.868042 2642 raft_consensus.cc:1261] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 1 FOLLOWER]: Refusing update from remote peer 95cfec1521b0443988bc7c8aab7c5f1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:38.868042 2494 raft_consensus.cc:1261] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 1 FOLLOWER]: Refusing update from remote peer 95cfec1521b0443988bc7c8aab7c5f1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:38.869143 2737 consensus_queue.cc:1035] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9741af9b9b4897ab069fd7f32e6553" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:38.869540 2725 consensus_queue.cc:1035] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "75bcc42b4ae541aca927d680420d13f3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:38.883667 2741 mvcc.cc:205] Tried to move back new op lower bound from 7316040904151179264 to 7316040903727149056. Current Snapshot: MvccSnapshot[applied={T|T < 7316040904151179264}]
I20260807 22:46:38.905061 2743 mvcc.cc:205] Tried to move back new op lower bound from 7316040904151179264 to 7316040903727149056. Current Snapshot: MvccSnapshot[applied={T|T < 7316040904151179264}]
I20260807 22:46:38.914119 2169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:38.914424 2169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:38.919183 2169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13fc4644c9884d71a081fd41da23397b: Sending DeleteTablet for 3 replicas of tablet 93691199a6ac4df5a8e019f92e353083
I20260807 22:46:38.921151 2620 tablet_service.cc:1514] Processing DeleteTablet for tablet 93691199a6ac4df5a8e019f92e353083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:38190
I20260807 22:46:38.921917 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet 93691199a6ac4df5a8e019f92e353083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:45964
I20260807 22:46:38.924733 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2209
I20260807 22:46:38.927040 2471 tablet_service.cc:1514] Processing DeleteTablet for tablet 93691199a6ac4df5a8e019f92e353083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:43658
I20260807 22:46:38.947505 2753 tablet_replica.cc:330] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: stopping tablet replica
I20260807 22:46:38.948271 2755 tablet_replica.cc:330] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: stopping tablet replica
I20260807 22:46:38.948334 2753 raft_consensus.cc:2229] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:38.948959 2752 tablet_replica.cc:330] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: stopping tablet replica
I20260807 22:46:38.949510 2755 raft_consensus.cc:2229] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:38.949751 2753 raft_consensus.cc:2258] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.950259 2752 raft_consensus.cc:2229] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:38.950887 2755 raft_consensus.cc:2258] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.950881 2752 raft_consensus.cc:2258] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.953320 2752 ts_tablet_manager.cc:1905] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:38.954427 2753 ts_tablet_manager.cc:1905] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:38.961158 2755 ts_tablet_manager.cc:1905] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:38.961814 2752 ts_tablet_manager.cc:1918] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:38.962095 2752 log.cc:1196] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/93691199a6ac4df5a8e019f92e353083
I20260807 22:46:38.963472 2752 ts_tablet_manager.cc:1939] T 93691199a6ac4df5a8e019f92e353083 P fc9741af9b9b4897ab069fd7f32e6553: Deleting consensus metadata
I20260807 22:46:38.964057 2753 ts_tablet_manager.cc:1918] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:38.964418 2753 log.cc:1196] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/93691199a6ac4df5a8e019f92e353083
I20260807 22:46:38.965138 2753 ts_tablet_manager.cc:1939] T 93691199a6ac4df5a8e019f92e353083 P 95cfec1521b0443988bc7c8aab7c5f1f: Deleting consensus metadata
I20260807 22:46:38.965767 2154 catalog_manager.cc:4841] TS fc9741af9b9b4897ab069fd7f32e6553 (127.20.10.131:43849): tablet 93691199a6ac4df5a8e019f92e353083 (table test-table [id=8b62d8d8802249c7a88ecb7082be71d6]) successfully deleted
I20260807 22:46:38.967413 2152 catalog_manager.cc:4841] TS 95cfec1521b0443988bc7c8aab7c5f1f (127.20.10.129:42633): tablet 93691199a6ac4df5a8e019f92e353083 (table test-table [id=8b62d8d8802249c7a88ecb7082be71d6]) successfully deleted
I20260807 22:46:38.979717 2755 ts_tablet_manager.cc:1918] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:38.980985 2755 log.cc:1196] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/93691199a6ac4df5a8e019f92e353083
I20260807 22:46:38.986683 2755 ts_tablet_manager.cc:1939] T 93691199a6ac4df5a8e019f92e353083 P 75bcc42b4ae541aca927d680420d13f3: Deleting consensus metadata
I20260807 22:46:38.989770 2152 catalog_manager.cc:4841] TS 75bcc42b4ae541aca927d680420d13f3 (127.20.10.130:42143): tablet 93691199a6ac4df5a8e019f92e353083 (table test-table [id=8b62d8d8802249c7a88ecb7082be71d6]) successfully deleted
I20260807 22:46:38.993320 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.926076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.20.10.129:42633)
0807 22:46:38.926247 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.926253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.962375 (+ 36122us) server_negotiation.cc:407] Connection header received
0807 22:46:38.962619 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.962625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.962753 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.962886 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.968745 (+ 5859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.970151 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.972642 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.973523 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.973684 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.984663 (+ 10979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.984688 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.984711 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:38.984784 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.989592 (+ 4808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.990638 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.990678 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.990683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.990923 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.991418 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.991423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.991427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.991858 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.992217 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.992630 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.992903 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:46:38.994869 2379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:39.107650 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2209
I20260807 22:46:39.134521 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2388
I20260807 22:46:39.151247 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.135482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.20.10.130:42143)
0807 22:46:39.135708 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.135715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.135794 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:46:39.136016 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.136022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.136103 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.136316 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:39.137761 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.138545 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.139938 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.140442 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.140598 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.143967 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:39.143987 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:39.144003 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:39.144054 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:39.147788 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:39.148611 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:39.148621 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:39.148623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:39.148864 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:39.149283 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:39.149289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:39.149292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:39.149867 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:39.150213 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.150556 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.150838 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:46:39.152208 2539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:39.248202 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2388
I20260807 22:46:39.275377 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2548
I20260807 22:46:39.292546 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.276430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.20.10.131:43849)
0807 22:46:39.276920 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.276925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.276944 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:46:39.277002 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.277006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.277079 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.277176 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:39.278753 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.279591 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.281048 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.281591 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.281706 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.285030 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:39.285048 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:39.285067 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:39.285239 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:39.289099 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:39.289944 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:39.289952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:39.289955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:39.290099 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:39.290581 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:39.290585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:39.290587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:39.291079 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:39.291440 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.291853 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.292053 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":267,"threads_started":1}
I20260807 22:46:39.293442 2688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:39.384994 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2548
I20260807 22:46:39.406308 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2133
I20260807 22:46:39.408036 2202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:39.530876 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2133
I20260807 22:46:39.561039 20522 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:46:39.908447 20522 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:46:39.908756 20522 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:46:40.266395 20522 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:46:40.266 UTC [1105] LOG: received smart shutdown request
2026-08-07 22:46:40.271 UTC [1105] LOG: background worker "logical replication launcher" (PID 1115) exited with exit code 1
2026-08-07 22:46:40.271 UTC [1109] LOG: shutting down
2026-08-07 22:46:40.488 UTC [1105] LOG: database system is shut down
2026-08-07T22:46:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (63090 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:40 dist-test-slave-csg5 krb5kdc[2769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:40 dist-test-slave-csg5 krb5kdc[2769](info): set up 2 sockets
Aug 07 22:46:40 dist-test-slave-csg5 krb5kdc[2769](info): commencing operation
krb5kdc: starting...
W20260807 22:46:41.871167 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:46:42 dist-test-slave-csg5 krb5kdc[2769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:46:42.210115 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:37575
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:42.587826 2782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:42.588240 2782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:42.588425 2782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:42.596452 2782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:42.596627 2782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:42.596732 2782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:42.596843 2782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:42.596930 2782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:42.608819 2782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:37575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:42.610528 2782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:42.612164 2782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:42.627620 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.627696 2788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:42.628535 2782 server_base.cc:1034] running on GCE node
W20260807 22:46:42.628741 2791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:42.629792 2782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:42.631685 2782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:42.632952 2782 hybrid_clock.cc:648] HybridClock initialized: now 1786142802632907 us; error 46 us; skew 500 ppm
I20260807 22:46:42.633450 2782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:42 dist-test-slave-csg5 krb5kdc[2769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:42.641343 2782 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:42.643401 2782 webserver.cc:460] Webserver started at http://127.20.10.190:33857/ using document root <none> and password file <none>
I20260807 22:46:42.644284 2782 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:42.644459 2782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:42.644822 2782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:42.648504 2782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "773d1ad679c24c52821715645f0b7346"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-csg5"
server_key: "9afaa8880be7460562731080eed01268"
server_key_iv: "138c1dc12df32916046d81c3cff29c2e"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.649539 2782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "773d1ad679c24c52821715645f0b7346"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-csg5"
server_key: "9afaa8880be7460562731080eed01268"
server_key_iv: "138c1dc12df32916046d81c3cff29c2e"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.655773 2782 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:46:42.660275 2798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.661983 2782 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:46:42.662232 2782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "773d1ad679c24c52821715645f0b7346"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-csg5"
server_key: "9afaa8880be7460562731080eed01268"
server_key_iv: "138c1dc12df32916046d81c3cff29c2e"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.662489 2782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:42.683415 2782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:42.689123 2782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:42.689507 2782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:42.712728 2782 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:37575
I20260807 22:46:42.712770 2851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:37575 every 8 connection(s)
I20260807 22:46:42.714272 2782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:42.715572 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2782
I20260807 22:46:42.715941 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:46:42.716854 20522 external_mini_cluster.cc:1428] Setting key b0d082a221cd6c2f48593aaac4fa3842
I20260807 22:46:42.720252 2852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:42.732127 2852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.10.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.189@KRBTEST.COM" created.
Aug 07 22:46:42 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:42.761844 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.735991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.20.10.190:37575)
0807 22:46:42.737044 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.737066 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.737784 (+ 718us) server_negotiation.cc:407] Connection header received
0807 22:46:42.740522 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.740533 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.740907 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.741153 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.742784 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.744926 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.746710 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.747763 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.748010 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.753504 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.753514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.753528 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:42.753569 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.756864 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:42.757639 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:42.757651 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:42.757655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:42.757828 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:42.758262 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:42.758270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:42.758274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:42.758915 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.759161 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.759741 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.760179 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":760,"thread_start_us":480,"threads_started":1}
I20260807 22:46:42.761883 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.735991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:37575 (local address 127.0.0.1:39890)
0807 22:46:42.737553 (+ 1562us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.737605 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.740030 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.741319 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.741325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.742031 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:42.742510 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.742533 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.745162 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.745166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.746493 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.746508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.748074 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.748080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.748360 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.749171 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:42.749198 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:42.753221 (+ 4023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:42.757023 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:42.757037 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:42.757051 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:42.757491 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:42.757962 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:42.757967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:42.757971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:42.758148 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:42.759085 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:42.759094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:42.759340 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.759625 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.760310 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1255,"mutex_wait_us":186,"spinlock_wait_cycles":74624,"thread_start_us":980,"threads_started":1}
I20260807 22:46:42.765175 2852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:46:42.766789 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.766161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42416)
0807 22:46:42.766412 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.766527 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260807 22:46:42.767247 2800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
I20260807 22:46:42.769162 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.768735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42420)
0807 22:46:42.768944 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.769017 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 22:46:42.769450 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:39895: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
I20260807 22:46:42.796694 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:39895
--webserver_interface=127.20.10.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:42.818886 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } attempt: 1
I20260807 22:46:42.820353 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.819796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42434)
0807 22:46:42.820027 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.820194 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 22:46:42.821491 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.821104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42446)
0807 22:46:42.821260 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.821345 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:46:42.821676 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:39895: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
I20260807 22:46:42.889957 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } attempt: 2
I20260807 22:46:42.891449 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.890817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42460)
0807 22:46:42.891121 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.891254 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 22:46:42.892782 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.892311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42462)
0807 22:46:42.892503 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.892599 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:46:42.893059 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:39895: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
I20260807 22:46:42.984378 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } attempt: 3
I20260807 22:46:42.985855 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.985290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42478)
0807 22:46:42.985536 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.985667 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 22:46:42.987221 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.986725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42490)
0807 22:46:42.986899 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.987017 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 22:46:42.987449 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:39895: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
I20260807 22:46:43.130788 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } attempt: 4
I20260807 22:46:43.132365 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.131646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42502)
0807 22:46:43.132072 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.132192 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246}
I20260807 22:46:43.133659 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.133265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42506)
0807 22:46:43.133423 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.133517 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:46:43.134094 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:39895: Network error: Client connection negotiation failed: client connection to 127.20.10.189:39895: connect: Connection refused (error 111)
W20260807 22:46:43.179461 2859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:43.179881 2859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:43.180059 2859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:43.189399 2859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:43.189616 2859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:43.189759 2859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:43.189886 2859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:43.189992 2859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:43.201776 2859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.10.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:43.203413 2859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:43.204887 2859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:43.218843 2866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:43.218811 2870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:43.218786 2867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:43.219710 2859 server_base.cc:1034] running on GCE node
I20260807 22:46:43.220678 2859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:43.222522 2859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:43.223834 2859 hybrid_clock.cc:648] HybridClock initialized: now 1786142803223753 us; error 84 us; skew 500 ppm
I20260807 22:46:43.224376 2859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:43.228597 2859 init.cc:377] Logged in from keytab as kudu/127.20.10.189@KRBTEST.COM (short username kudu)
I20260807 22:46:43.230561 2859 webserver.cc:460] Webserver started at http://127.20.10.189:43875/ using document root <none> and password file <none>
I20260807 22:46:43.231434 2859 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:43.231609 2859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:43.231951 2859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:43.235460 2859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data/instance:
uuid: "f7a84f8e113447d6ae962791c5dda8d2"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "1b9a393e5b0fe995d25119d4b3c8cbb5"
server_key_iv: "1dd74f638de55fbcf987e45237f97a25"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.236495 2859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal/instance:
uuid: "f7a84f8e113447d6ae962791c5dda8d2"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "1b9a393e5b0fe995d25119d4b3c8cbb5"
server_key_iv: "1dd74f638de55fbcf987e45237f97a25"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.242726 2859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:46:43.247676 2877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:43.249366 2859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:43.249599 2859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "f7a84f8e113447d6ae962791c5dda8d2"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "1b9a393e5b0fe995d25119d4b3c8cbb5"
server_key_iv: "1dd74f638de55fbcf987e45237f97a25"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.249856 2859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:43.273913 2859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:43.278745 2859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:43.279085 2859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:43.302193 2859 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.189:39895
I20260807 22:46:43.302232 2934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.189:39895 every 8 connection(s)
I20260807 22:46:43.303809 2859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
I20260807 22:46:43.308115 2935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:43.310528 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2859
I20260807 22:46:43.310781 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-1/wal/instance
I20260807 22:46:43.311268 20522 external_mini_cluster.cc:1428] Setting key 31b013147125c3bff87b33fe99e2e19f
I20260807 22:46:43.316119 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.10.188@KRBTEST.COM; defaulting to no policy
I20260807 22:46:43.342393 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.319139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.20.10.190:37575)
0807 22:46:43.319798 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.319807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.320887 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 22:46:43.323057 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.323067 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.323173 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.323329 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.326089 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.327090 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.330067 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.330523 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.330638 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.335788 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.335793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.335797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.335852 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.339005 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.339868 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.339876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.339880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.340016 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.340481 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.340486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.340489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.340789 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.340968 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.341725 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.341975 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":335,"threads_started":1}
I20260807 22:46:43.343535 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.319791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:37575 (local address 127.0.0.1:39906)
0807 22:46:43.320686 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.320742 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.322752 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.323715 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.323731 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.324707 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.325823 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.325855 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.327282 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.327292 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.329853 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.329875 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.330839 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.330850 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.331238 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.332158 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:43.332209 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:43.335485 (+ 3276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:43.339184 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.339194 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.339214 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.339720 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.340137 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.340144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.340147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.340352 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.340931 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:43.340940 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:43.341345 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 22:46:43.341653 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 22:46:43.342221 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":398,"threads_started":1}
I20260807 22:46:43.345603 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.188@KRBTEST.COM" created.
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:46:43.361675 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.346962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42510)
0807 22:46:43.347144 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.347165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.347304 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.348250 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.348254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.348741 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.348891 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.348899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.350314 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.350320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.351266 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.351279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.352195 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.352200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.352425 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.353069 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:43.353087 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:43.355491 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:43.358937 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.358945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.358949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.359348 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.359771 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.359776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.359779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.359929 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.360561 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:43.360566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:43.360717 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 22:46:43.360874 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:46:43.361183 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:46:43.361737 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.347051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.20.10.189:39895)
0807 22:46:43.347752 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.347758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.347784 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:46:43.347836 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.347844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.348086 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.348262 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.349073 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.350175 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.351408 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.352029 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.352166 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.355720 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.355732 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.355746 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.355782 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.358802 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.359497 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.359506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.359509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.359653 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.360053 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.360057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.360061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.360438 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.360665 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.361035 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.361305 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":512,"thread_start_us":357,"threads_started":1}
I20260807 22:46:43.364486 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:46:43.365778 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.365232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51654)
0807 22:46:43.365449 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.365578 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 22:46:43.366108 2878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.369144 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.368699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51666)
0807 22:46:43.368908 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.368999 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:46:43.369374 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
I20260807 22:46:43.389595 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:46577
--webserver_interface=127.20.10.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:43.418854 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 1
I20260807 22:46:43.420528 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.419766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51678)
0807 22:46:43.420016 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.420138 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 22:46:43.422886 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.421612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51684)
0807 22:46:43.422115 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.422637 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245}
W20260807 22:46:43.423136 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.433622 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } attempt: 5
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:46:43.453636 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.435021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.20.10.189:39895)
0807 22:46:43.435389 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.435396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.435425 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:46:43.435584 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.435590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.435710 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.435867 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.436880 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.438417 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.440192 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.440858 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.441009 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.445134 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.445162 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.445170 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.445236 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.449774 (+ 4538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.450986 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.450994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.450997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.451282 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.451806 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.451811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.451815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.452258 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.452453 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.452973 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.453275 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260807 22:46:43.453933 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.434912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42518)
0807 22:46:43.435167 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.435308 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.435444 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.435958 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.435970 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.436401 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.436606 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.436616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.438865 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.438871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.439973 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.439999 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.441112 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.441118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.441285 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.442041 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:43.442062 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:43.444481 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:43.450097 (+ 5616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.450103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.450106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.450787 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.451468 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.451473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.451475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.451647 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.452455 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:43.452459 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:43.452650 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 22:46:43.452801 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:46:43.453107 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 22:46:43.455533 2852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:46:43.457273 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.456738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51698)
0807 22:46:43.456977 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.457080 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 22:46:43.458487 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.458063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51702)
0807 22:46:43.458245 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.458354 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:46:43.458758 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.491426 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 2
I20260807 22:46:43.492882 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.492263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51708)
0807 22:46:43.492529 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.492664 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 22:46:43.494159 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.493719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51716)
0807 22:46:43.493899 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.493994 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 22:46:43.494505 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.510051 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 1
I20260807 22:46:43.511503 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.510960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51722)
0807 22:46:43.511205 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.511309 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 22:46:43.512862 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.512425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51734)
0807 22:46:43.512631 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.512722 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 22:46:43.513131 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.581413 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 2
I20260807 22:46:43.582828 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.582218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51746)
0807 22:46:43.582480 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.582573 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:46:43.583945 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.583571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51756)
0807 22:46:43.583754 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.583827 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:46:43.584210 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.585846 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 3
I20260807 22:46:43.587144 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.586553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51768)
0807 22:46:43.586881 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.586973 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260807 22:46:43.588416 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.587869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51776)
0807 22:46:43.588208 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.588296 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260807 22:46:43.588640 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.690474 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 3
I20260807 22:46:43.691965 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.691260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51784)
0807 22:46:43.691565 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.691710 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 22:46:43.693571 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.692958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51786)
0807 22:46:43.693222 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.693370 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260807 22:46:43.693902 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.731863 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 4
I20260807 22:46:43.733343 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.732705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51798)
0807 22:46:43.733052 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.733167 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260807 22:46:43.734543 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.734163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51812)
0807 22:46:43.734338 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.734420 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 22:46:43.734849 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
W20260807 22:46:43.812039 2943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:43.812364 2943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:43.812528 2943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:43.820503 2943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:43.820638 2943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:43.820724 2943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:43.820816 2943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:43.820930 2943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:43.832289 2943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:37575,127.20.10.189:39895,127.20.10.188:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:46577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.10.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:43.833810 2943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:43.835311 2943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:43.849185 2948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:43.849323 2949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:43.849231 2951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:43.850538 2943 server_base.cc:1034] running on GCE node
I20260807 22:46:43.851567 2943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:43.853376 2943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:43.854619 2943 hybrid_clock.cc:648] HybridClock initialized: now 1786142803854554 us; error 66 us; skew 500 ppm
I20260807 22:46:43.855165 2943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:43.859302 2943 init.cc:377] Logged in from keytab as kudu/127.20.10.188@KRBTEST.COM (short username kudu)
I20260807 22:46:43.861243 2943 webserver.cc:460] Webserver started at http://127.20.10.188:34895/ using document root <none> and password file <none>
I20260807 22:46:43.862103 2943 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:43.862277 2943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:43.862643 2943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:43.866257 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data/instance:
uuid: "5a73fe518d9e454eb54f784f07288c2f"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "2664086d2198a1bc39e5a9c655a2040d"
server_key_iv: "90ffa5e1aadff23a0a1af6fe88059f24"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.867309 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal/instance:
uuid: "5a73fe518d9e454eb54f784f07288c2f"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "2664086d2198a1bc39e5a9c655a2040d"
server_key_iv: "90ffa5e1aadff23a0a1af6fe88059f24"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.871172 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 4
I20260807 22:46:43.872536 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.871908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51822)
0807 22:46:43.872212 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.872351 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 22:46:43.873806 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.873350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51834)
0807 22:46:43.873526 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.873627 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 22:46:43.874094 2943 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
W20260807 22:46:43.874200 2852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:46577: Network error: Client connection negotiation failed: client connection to 127.20.10.188:46577: connect: Connection refused (error 111)
I20260807 22:46:43.878268 2958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:43.880210 2943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:46:43.880458 2943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "5a73fe518d9e454eb54f784f07288c2f"
format_stamp: "Formatted at 2026-08-07 22:46:43 on dist-test-slave-csg5"
server_key: "2664086d2198a1bc39e5a9c655a2040d"
server_key_iv: "90ffa5e1aadff23a0a1af6fe88059f24"
server_key_version: "encryptionkey@0"
I20260807 22:46:43.880766 2943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:43.901856 2943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:43.906991 2943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:43.907325 2943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:43.928146 2943 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.188:46577
I20260807 22:46:43.928210 3011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.188:46577 every 8 connection(s)
I20260807 22:46:43.929755 2943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
I20260807 22:46:43.934192 3012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:43.937439 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 2943
I20260807 22:46:43.937656 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142560216906-20522-0/minicluster-data/master-2/wal/instance
I20260807 22:46:43.938205 20522 external_mini_cluster.cc:1428] Setting key 0c4e22470bb28b9613cf83ec7f882e27
I20260807 22:46:43.944650 3012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:43.968983 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.947998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.20.10.190:37575)
0807 22:46:43.949615 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.949622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.949642 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:46:43.949702 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.949708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.949817 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.949949 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.951619 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.952696 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.954873 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.955428 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.955554 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.959794 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.959805 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.959812 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.959872 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.963886 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.965023 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.965030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.965033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.965254 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.965838 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.965842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.965845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.966376 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.966547 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.968236 (+ 1689us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.968475 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1486,"thread_start_us":1334,"threads_started":1}
I20260807 22:46:43.969741 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.947914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:37575 (local address 127.0.0.1:39918)
0807 22:46:43.948862 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.948923 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.949081 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.950027 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.950033 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.951028 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.951432 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.951446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.952904 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.952907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.954641 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.954683 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.955747 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.955758 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.955989 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.956760 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:43.956800 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:43.959515 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:43.964054 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.964068 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.964082 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.964829 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.965409 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.965412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.965415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.965691 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.966605 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:43.966611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:43.967071 (+ 460us) client_negotiation.cc:769] Sending connection context
0807 22:46:43.967609 (+ 538us) client_negotiation.cc:240] Negotiation successful
0807 22:46:43.968271 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"spinlock_wait_cycles":22912,"thread_start_us":475,"threads_started":1}
I20260807 22:46:43.969774 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.940579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.20.10.188:46577)
0807 22:46:43.941644 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.941663 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.941732 (+ 69us) server_negotiation.cc:407] Connection header received
0807 22:46:43.944520 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.944551 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.945177 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.945754 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.947623 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.949990 (+ 2367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.952140 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.953207 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.953362 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.959133 (+ 5771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.959144 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.959163 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.959212 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.963798 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.965031 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.965041 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.965044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.965387 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.965952 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.965959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.965962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.967005 (+ 1043us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.967446 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.968215 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.968517 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":484,"threads_started":1}
I20260807 22:46:43.972373 3012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:43.991496 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.973305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:39895 (local address 127.0.0.1:42532)
0807 22:46:43.973579 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.973597 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.973733 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.974479 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.974485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.975264 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.975526 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.975564 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.977498 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.977504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.978961 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.978976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.980073 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.980081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.980526 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.981328 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:43.981348 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:43.984019 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:43.988466 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.988475 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.988481 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.989089 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.989561 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:43.989566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:43.989571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:43.989753 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:43.990312 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:43.990317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:43.990508 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 22:46:43.990693 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 22:46:43.991054 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 22:46:43.991564 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.973437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.20.10.189:39895)
0807 22:46:43.974087 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.974094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.974112 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:46:43.974175 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.974180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.974269 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.974390 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.975798 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.977293 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.979134 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.979884 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.979991 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.984309 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.984318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.984323 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.984373 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.988243 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.989244 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.989249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.989253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.989430 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.989879 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.989883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.989885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.990171 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.990370 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.990829 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.991069 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":364,"threads_started":1}
I20260807 22:46:43.992763 3012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:46:43.993872 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.973192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.20.10.190:37575)
0807 22:46:43.973416 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.973423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.973610 (+ 187us) server_negotiation.cc:407] Connection header received
0807 22:46:43.973887 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.973893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.973992 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.974071 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.976296 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.977481 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.979703 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.980293 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.980447 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.986408 (+ 5961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:43.986414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:43.986418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:43.986495 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:43.990098 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.991019 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.991023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.991026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.991161 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:43.991563 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:43.991584 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:43.991590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:43.992461 (+ 871us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:43.992770 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.993105 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.993352 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 22:46:43 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:46:44 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:46:44.010509 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.993640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51848)
0807 22:46:43.993843 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.993867 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:43.993978 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:43.994348 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:43.994355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:43.994827 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:43.995034 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.995046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.996604 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.996611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.998067 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:43.998090 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.999243 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.999247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:43.999444 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.000107 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:44.000132 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:44.002653 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:44.007158 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.007167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.007173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.007765 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.008284 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.008289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.008293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.008478 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.009421 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:44.009428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:44.009652 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 22:46:44.009823 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 22:46:44.010098 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 22:46:44.011157 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.993808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.20.10.188:46577)
0807 22:46:43.994008 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.994016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.994040 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:46:43.994095 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.994100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.994169 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.994283 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.995204 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.996461 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.998294 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.999075 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.999207 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.002924 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.002934 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.002941 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.002984 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.006987 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.007916 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.007920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.007923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.008050 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.008603 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.008609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.008612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.009092 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.009415 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.009990 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.010427 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:46:44.016280 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.995971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.20.10.189:39895)
0807 22:46:43.996172 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.996179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.996199 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:46:43.996251 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.996255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.996332 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.996442 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:43.998068 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.999308 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.001331 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.001803 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.001981 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.008511 (+ 6530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.008518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.008552 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.008607 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.012383 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.013426 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.013430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.013433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.013653 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.014368 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.014372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.014374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.014797 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.015029 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.015618 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.015842 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:46:44.017786 3012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: Bootstrap starting.
I20260807 22:46:44.021461 3012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:44.021526 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:44.022776 3012 log.cc:824] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:44.026508 3012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: No bootstrap required, opened a new log
I20260807 22:46:44.034098 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 5
I20260807 22:46:44.040704 3012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.041507 3012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:44.041751 3012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a73fe518d9e454eb54f784f07288c2f, State: Initialized, Role: FOLLOWER
I20260807 22:46:44.042822 3012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.055685 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.026905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.20.10.189:39895)
0807 22:46:44.027554 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.027569 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.031419 (+ 3850us) server_negotiation.cc:407] Connection header received
0807 22:46:44.031782 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.031790 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.031905 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.031983 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.035257 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.036473 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.038989 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.039702 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.039809 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.044029 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.044041 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.044049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.044115 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.049159 (+ 5044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.051116 (+ 1957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.051125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.051129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.051340 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.053075 (+ 1735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.053084 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.053090 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.053635 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.053942 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.054841 (+ 899us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.055199 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483}
I20260807 22:46:44.059302 3028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.063984 3028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 22:46:44 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142803, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
I20260807 22:46:44.064337 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.031810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.20.10.188:46577)
0807 22:46:44.032020 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.032026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.034186 (+ 2160us) server_negotiation.cc:407] Connection header received
0807 22:46:44.034534 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.034542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.034647 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.034783 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.038986 (+ 4203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.040333 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.044979 (+ 4646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.046157 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.046291 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.052013 (+ 5722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.052024 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.052032 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.052086 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.058893 (+ 6807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.060095 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.060101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.060104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.060315 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.060861 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.060869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.060872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.061431 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.061757 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.063281 (+ 1524us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.063556 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:46:44.064581 3028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:44.064922 3028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.067031 3012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:44.067497 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.027127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.20.10.190:37575)
0807 22:46:44.027342 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.027348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.031302 (+ 3954us) server_negotiation.cc:407] Connection header received
0807 22:46:44.031798 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.031804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.031894 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.031984 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.038448 (+ 6464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.039776 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.049187 (+ 9411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.050137 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.050298 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.058634 (+ 8336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.058643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.058647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.058742 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.063467 (+ 4725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.064505 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.064513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.064521 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.064723 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.065292 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.065297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.065301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.065734 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.066047 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.066759 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.067037 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:46:44.069707 3028 leader_election.cc:288] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 773d1ad679c24c52821715645f0b7346 (127.20.10.190:37575), f7a84f8e113447d6ae962791c5dda8d2 (127.20.10.189:39895)
I20260807 22:46:44.075157 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.035521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51874)
0807 22:46:44.035830 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:44.035853 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:44.035993 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:44.051695 (+ 15702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:44.051703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:44.052370 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:44.052670 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.052684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.055447 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.055455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.057403 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.057426 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.060510 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.060517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.060765 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.061463 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:44.061485 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:44.065312 (+ 3827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:44.071556 (+ 6244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.071562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.071565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.072421 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.072928 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.072936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.072940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.073143 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.074034 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:44.074039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:44.074220 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 22:46:44.074393 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:46:44.074749 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260807 22:46:44.075235 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.035041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.20.10.188:46577)
0807 22:46:44.051211 (+ 16170us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.051220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.051252 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:46:44.051330 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.051338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.051447 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.052876 (+ 1429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.053330 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.055162 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.057681 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.060325 (+ 2644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.061073 (+ 748us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.066001 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.066012 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.066020 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.066069 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.071356 (+ 5287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.072541 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.072548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.072551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.072722 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.073354 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.073361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.073366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.073867 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.074057 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.074509 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.074774 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15923,"thread_start_us":16673,"threads_started":1}
I20260807 22:46:44.082988 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: Bootstrap starting.
W20260807 22:46:44.084389 3041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:44.084553 3041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:44.092061 2935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:44.093801 2935 log.cc:824] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:44.086047 2943 master_runner.cc:186] Error getting master registration for 127.20.10.190:37575: OK, instance_id { permanent_uuid: "773d1ad679c24c52821715645f0b7346" instance_seqno: 1786142802706164 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:46:44.098403 2943 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:46:44.108426 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: No bootstrap required, opened a new log
I20260807 22:46:44.123534 2935 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.124104 2935 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:44.124243 2935 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a84f8e113447d6ae962791c5dda8d2, State: Initialized, Role: FOLLOWER
I20260807 22:46:44.125041 2935 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.128079 3044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.128722 3044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.129731 2935 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:44.129791 2907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a73fe518d9e454eb54f784f07288c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a84f8e113447d6ae962791c5dda8d2" is_pre_election: true
I20260807 22:46:44.130565 2907 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a73fe518d9e454eb54f784f07288c2f in term 0.
I20260807 22:46:44.131520 2960 leader_election.cc:302] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a73fe518d9e454eb54f784f07288c2f, f7a84f8e113447d6ae962791c5dda8d2; no voters:
I20260807 22:46:44.132122 3029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:44.132335 3029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:44.132530 3029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:44.136142 3029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.139372 2907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a73fe518d9e454eb54f784f07288c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a84f8e113447d6ae962791c5dda8d2"
I20260807 22:46:44.139668 2907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:44.142457 2907 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a73fe518d9e454eb54f784f07288c2f in term 1.
I20260807 22:46:44.142901 3029 leader_election.cc:288] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [CANDIDATE]: Term 1 election: Requested vote from peers 773d1ad679c24c52821715645f0b7346 (127.20.10.190:37575), f7a84f8e113447d6ae962791c5dda8d2 (127.20.10.189:39895)
I20260807 22:46:44.143957 2960 leader_election.cc:302] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a73fe518d9e454eb54f784f07288c2f, f7a84f8e113447d6ae962791c5dda8d2; no voters:
I20260807 22:46:44.144369 3029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:44.144594 3029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [term 1 LEADER]: Becoming Leader. State: Replica: 5a73fe518d9e454eb54f784f07288c2f, State: Running, Role: LEADER
I20260807 22:46:44.145246 3029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.149694 3028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a73fe518d9e454eb54f784f07288c2f. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.150136 3028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:44.151472 2852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } attempt: 5
I20260807 22:46:44.156483 3051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:44.159468 3051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 22:46:44.159770 2964 tablet.cc:2369] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:46:44 dist-test-slave-csg5 krb5kdc[2769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142802, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
I20260807 22:46:44.176920 2907 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [term 1 FOLLOWER]: Refusing update from remote peer 5a73fe518d9e454eb54f784f07288c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:44.178295 3028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:44.180703 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.152416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:46577 (local address 127.0.0.1:51880)
0807 22:46:44.153080 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:44.153113 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:44.153244 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:44.155073 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:44.155082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:44.155799 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:44.156077 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.156091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.157894 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.157902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.159565 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.159585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.160904 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.160912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.161117 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.161883 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:44.161909 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:44.165010 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:44.177076 (+ 12066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.177084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.177088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.177778 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.178423 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.178429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.178433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.178685 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.179531 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:44.179536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:44.179740 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 22:46:44.179889 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 22:46:44.180247 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524}
I20260807 22:46:44.182197 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.154130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.20.10.188:46577)
0807 22:46:44.154363 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.154372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.154402 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:46:44.154472 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.154480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.154618 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.154776 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.156299 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.157670 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.159779 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.160498 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.160645 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.165340 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.165352 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.165364 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.165417 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.176836 (+ 11419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.178024 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.178032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.178042 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.178246 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.178865 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.178871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.178874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.179363 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.181027 (+ 1664us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.181243 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.181590 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:46:44.190521 2852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: Bootstrap starting.
I20260807 22:46:44.197145 2852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:44.172520 2859 master_runner.cc:186] Error getting master registration for 127.20.10.190:37575: OK, instance_id { permanent_uuid: "773d1ad679c24c52821715645f0b7346" instance_seqno: 1786142802706164 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:46:44.198993 2859 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:46:44.198949 2852 log.cc:824] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: Log is configured to *not* fsync() on all Append() calls
W20260807 22:46:44.205696 3059 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:44.205986 3059 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:44.220580 3044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a73fe518d9e454eb54f784f07288c2f. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.220970 3044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.221969 2852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: No bootstrap required, opened a new log
I20260807 22:46:44.232571 3055 mvcc.cc:205] Tried to move back new op lower bound from 7316040925891239936 to 7316040925784870912. Current Snapshot: MvccSnapshot[applied={T|T < 7316040925891239936}]
I20260807 22:46:44.236224 2852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.236981 2852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:44.237187 2852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 773d1ad679c24c52821715645f0b7346, State: Initialized, Role: FOLLOWER
I20260807 22:46:44.238127 2852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } }
I20260807 22:46:44.242414 3044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.242808 3044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.248484 3029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.248903 3029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:44.260053 3051 catalog_manager.cc:1347] Generated new cluster ID: 5956f59cd4ed4e529ccc0767641fb619
I20260807 22:46:44.260267 3051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:44.271405 3067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.271946 3067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.272993 2852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:44.273505 2825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:44.278880 2825 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [term 1 FOLLOWER]: Refusing update from remote peer 5a73fe518d9e454eb54f784f07288c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:44.274277 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a73fe518d9e454eb54f784f07288c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "773d1ad679c24c52821715645f0b7346"
I20260807 22:46:44.273837 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a73fe518d9e454eb54f784f07288c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "773d1ad679c24c52821715645f0b7346" is_pre_election: true
I20260807 22:46:44.282295 3028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:44.311655 3051 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:44.315007 3051 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:44.325205 3028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.325623 3028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:44.334694 3067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a73fe518d9e454eb54f784f07288c2f. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.335258 3067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.335681 3078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a73fe518d9e454eb54f784f07288c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "773d1ad679c24c52821715645f0b7346" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 37575 } } peers { permanent_uuid: "f7a84f8e113447d6ae962791c5dda8d2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 39895 } } peers { permanent_uuid: "5a73fe518d9e454eb54f784f07288c2f" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 46577 } } }
I20260807 22:46:44.336342 3078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:46:44.391217 3085 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:46:44.405151 3051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a73fe518d9e454eb54f784f07288c2f: Generated new TSK 0
I20260807 22:46:44.406394 3051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:45.209079 3059 catalog_manager.cc:1259] Loaded cluster ID: 5956f59cd4ed4e529ccc0767641fb619
I20260807 22:46:45.209220 3059 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: loading cluster ID for follower catalog manager: success
I20260807 22:46:45.212805 3059 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: acquiring CA information for follower catalog manager: success
I20260807 22:46:45.214968 3059 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f7a84f8e113447d6ae962791c5dda8d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:46:45.396245 3085 catalog_manager.cc:1259] Loaded cluster ID: 5956f59cd4ed4e529ccc0767641fb619
I20260807 22:46:45.396389 3085 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: loading cluster ID for follower catalog manager: success
I20260807 22:46:45.399425 3085 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: acquiring CA information for follower catalog manager: success
I20260807 22:46:45.401576 3085 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 773d1ad679c24c52821715645f0b7346: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:46:45.515389 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2782
I20260807 22:46:45.516970 2847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:45.628628 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2782
I20260807 22:46:45.653277 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2859
I20260807 22:46:45.654523 2930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:45.776223 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2859
I20260807 22:46:45.807380 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 2943
I20260807 22:46:45.808820 3007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:45.930781 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.929747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:37575 (local address 127.0.0.1:39954)
0807 22:46:45.930346 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.930518 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":260,"threads_started":1}
I20260807 22:46:45.930735 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 2943
2026-08-07T22:46:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5302 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:46 dist-test-slave-csg5 krb5kdc[3099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:46 dist-test-slave-csg5 krb5kdc[3099](info): set up 2 sockets
Aug 07 22:46:46 dist-test-slave-csg5 krb5kdc[3099](info): commencing operation
krb5kdc: starting...
W20260807 22:46:47.213994 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.005s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:46:47 dist-test-slave-csg5 krb5kdc[3099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:46:47.575383 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:41857
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:41857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:47.950016 3112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:47.950381 3112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:47.950551 3112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:47.959277 3112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:47.959430 3112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:47.959481 3112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:47.959534 3112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:47.959582 3112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:47.970818 3112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:41857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:41857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:47.972271 3112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:47.973722 3112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:47.987326 3118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:47.987416 3120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:47.987286 3117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:47.988242 3112 server_base.cc:1034] running on GCE node
I20260807 22:46:47.989154 3112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:47.990967 3112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:47.992295 3112 hybrid_clock.cc:648] HybridClock initialized: now 1786142807992190 us; error 41 us; skew 500 ppm
I20260807 22:46:47.992934 3112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:47 dist-test-slave-csg5 krb5kdc[3099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:48.002530 3112 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:48.004684 3112 webserver.cc:460] Webserver started at http://127.20.10.190:38413/ using document root <none> and password file <none>
I20260807 22:46:48.005558 3112 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:48.005735 3112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:48.006139 3112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:48.009807 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "120851694c60447880cc9e94a76b5b30"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-csg5"
server_key: "10ca7c7b17a13b4a5621e54d0fa2780b"
server_key_iv: "63abe5654eefbd29ce79623b37631ee2"
server_key_version: "encryptionkey@0"
I20260807 22:46:48.010820 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "120851694c60447880cc9e94a76b5b30"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-csg5"
server_key: "10ca7c7b17a13b4a5621e54d0fa2780b"
server_key_iv: "63abe5654eefbd29ce79623b37631ee2"
server_key_version: "encryptionkey@0"
I20260807 22:46:48.017081 3112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:46:48.021258 3127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:48.022993 3112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:48.023224 3112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "120851694c60447880cc9e94a76b5b30"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-csg5"
server_key: "10ca7c7b17a13b4a5621e54d0fa2780b"
server_key_iv: "63abe5654eefbd29ce79623b37631ee2"
server_key_version: "encryptionkey@0"
I20260807 22:46:48.023466 3112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:48.058277 3112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:48.063508 3112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:48.063861 3112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:48.085994 3112 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:41857
I20260807 22:46:48.086015 3184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:41857 every 8 connection(s)
I20260807 22:46:48.087653 3112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:48.091665 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3112
I20260807 22:46:48.091934 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:46:48.092044 3185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:48.092833 20522 external_mini_cluster.cc:1428] Setting key 3ae056513d8b11607c0bcf6725885221
I20260807 22:46:48.109881 3185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30: Bootstrap starting.
Aug 07 22:46:48 dist-test-slave-csg5 krb5kdc[3099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:48.114626 3185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:48.116328 3185 log.cc:824] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:48.121398 3185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30: No bootstrap required, opened a new log
I20260807 22:46:48.123925 3189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:48.095042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.20.10.190:41857)
0807 22:46:48.096051 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:48.096064 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:48.096115 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:46:48.097797 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:48.097822 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:48.098270 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:48.098645 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:48.101021 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:48.102468 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:48.104726 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:48.105551 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:48.105724 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:48.112008 (+ 6284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:48.112029 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:48.112053 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:48.112125 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:48.116917 (+ 4792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:48.118564 (+ 1647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:48.118578 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:48.118702 (+ 124us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:48.118986 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:48.119667 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:48.119675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:48.119678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:48.120520 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:48.121037 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:48.121630 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 22:46:48.122252 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":495,"threads_started":1}
I20260807 22:46:48.130007 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:48.133813 3185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } }
I20260807 22:46:48.134445 3185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:48.134568 3185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120851694c60447880cc9e94a76b5b30, State: Initialized, Role: FOLLOWER
I20260807 22:46:48.135279 3185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } }
I20260807 22:46:48.135521 3185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:48.135627 3185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:48.135771 3185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:48.138139 3185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } }
I20260807 22:46:48.138821 3185 leader_election.cc:302] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 120851694c60447880cc9e94a76b5b30; no voters:
I20260807 22:46:48.139544 3185 leader_election.cc:288] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:48.139778 3192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:48.140638 3192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [term 1 LEADER]: Becoming Leader. State: Replica: 120851694c60447880cc9e94a76b5b30, State: Running, Role: LEADER
I20260807 22:46:48.141280 3192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } }
I20260807 22:46:48.142019 3185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:48.144812 3193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "120851694c60447880cc9e94a76b5b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } } }
I20260807 22:46:48.145478 3193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:48.147541 3201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:48.147368 3194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 120851694c60447880cc9e94a76b5b30. Latest consensus state: current_term: 1 leader_uuid: "120851694c60447880cc9e94a76b5b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120851694c60447880cc9e94a76b5b30" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41857 } } }
I20260807 22:46:48.148062 3194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:48.152549 3201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:48.162314 3201 catalog_manager.cc:1347] Generated new cluster ID: d8ee0a07ee9a49b7a30b7b9aea188540
I20260807 22:46:48.162462 3201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:48.177277 3201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:48.179042 3201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:48.199885 3201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 120851694c60447880cc9e94a76b5b30: Generated new TSK 0
I20260807 22:46:48.201113 3201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:48.246901 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3112
I20260807 22:46:48.248389 3180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:48.353456 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3112
2026-08-07T22:46:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2417 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:48 dist-test-slave-csg5 krb5kdc[3215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:48 dist-test-slave-csg5 krb5kdc[3215](info): set up 2 sockets
Aug 07 22:46:48 dist-test-slave-csg5 krb5kdc[3215](info): commencing operation
krb5kdc: starting...
W20260807 22:46:49.619374 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:46:49 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:46:49.975769 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:50.372105 3229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:50.372454 3229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:50.372550 3229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:50.380436 3229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:50.380566 3229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:50.380615 3229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:50.380686 3229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:50.380746 3229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:50.391499 3229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:50.392930 3229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:50.394294 3229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:50.407603 3234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:50.407649 3235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:50.408143 3237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:50.409684 3229 server_base.cc:1034] running on GCE node
I20260807 22:46:50.410808 3229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:50.412617 3229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:50.413933 3229 hybrid_clock.cc:648] HybridClock initialized: now 1786142810413894 us; error 59 us; skew 500 ppm
I20260807 22:46:50.414377 3229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:50 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142810, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:50.418404 3229 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:50.420487 3229 webserver.cc:460] Webserver started at http://127.20.10.190:33585/ using document root <none> and password file <none>
I20260807 22:46:50.421422 3229 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:50.421612 3229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:50.421999 3229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:50.425877 3229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "aa49b0a062de43059eeb3d7769ffa141"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-csg5"
server_key: "652a4b21e954ef90783332d8d702b420"
server_key_iv: "defdf3afc97a355d5d8e030cfb8da2c7"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.427062 3229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "aa49b0a062de43059eeb3d7769ffa141"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-csg5"
server_key: "652a4b21e954ef90783332d8d702b420"
server_key_iv: "defdf3afc97a355d5d8e030cfb8da2c7"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.434136 3229 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 22:46:50.438942 3245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:50.440786 3229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 22:46:50.441066 3229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "aa49b0a062de43059eeb3d7769ffa141"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-csg5"
server_key: "652a4b21e954ef90783332d8d702b420"
server_key_iv: "defdf3afc97a355d5d8e030cfb8da2c7"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.441331 3229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:50.476523 3229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:50.481765 3229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:50.482111 3229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:50.504688 3229 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33961
I20260807 22:46:50.504740 3300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33961 every 8 connection(s)
I20260807 22:46:50.506248 3229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:50.510746 3301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:50.512722 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3229
I20260807 22:46:50.513003 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:46:50.513782 20522 external_mini_cluster.cc:1428] Setting key 4f00610bc37ec5ba521918f2fd289e0a
I20260807 22:46:50.525453 3301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap starting.
I20260807 22:46:50.530180 3301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Neither blocks nor log segments found. Creating new log.
Aug 07 22:46:50 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:50.531669 3301 log.cc:824] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:50.535908 3301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: No bootstrap required, opened a new log
I20260807 22:46:50.542270 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:50.516125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.20.10.190:33961)
0807 22:46:50.517114 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:50.517131 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:50.517176 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:46:50.519041 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:50.519067 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:50.519487 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:50.519830 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:50.521980 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:50.523630 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:50.525898 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:50.526759 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:50.526913 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:50.532770 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:50.532779 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:50.532797 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:50.532837 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:50.536800 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:50.538069 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:50.538077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:50.538088 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:50.538333 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:50.538943 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:50.538947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:50.538949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:50.539751 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:50.540010 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:50.540544 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 22:46:50.540927 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":456,"threads_started":1}
I20260807 22:46:50.546933 3301 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:50.547432 3301 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:50.547567 3301 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Initialized, Role: FOLLOWER
I20260807 22:46:50.548260 3301 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:50.548548 3301 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:50.548724 3301 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:50.548938 3301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:50.551616 3301 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:50.552249 3301 leader_election.cc:302] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa49b0a062de43059eeb3d7769ffa141; no voters:
I20260807 22:46:50.553030 3301 leader_election.cc:288] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:50.553262 3307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:50.554205 3307 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 LEADER]: Becoming Leader. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Running, Role: LEADER
I20260807 22:46:50.555003 3307 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:50.555809 3301 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:50.558632 3308 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:50.558568 3309 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa49b0a062de43059eeb3d7769ffa141. Latest consensus state: current_term: 1 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:50.559221 3308 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:50.559293 3309 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:50.564672 3316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:50.570972 3316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:50.581796 3316 catalog_manager.cc:1347] Generated new cluster ID: 0c380c4d449944c6a4d7d2652a858b19
I20260807 22:46:50.581959 3316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:50.603865 3316 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:50.605926 3316 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:50.626389 3316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Generated new TSK 0
I20260807 22:46:50.627715 3316 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:46:50.723678 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:51.123301 3333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:51.123660 3333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:51.123834 3333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:51.133153 3333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:51.133584 3333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:51.133831 3333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:46:51.145692 3333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:51.147420 3333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:51.149219 3333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:51.164392 3339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:51.164556 3338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:51.164590 3341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:51.165481 3333 server_base.cc:1034] running on GCE node
I20260807 22:46:51.166249 3333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:51.167430 3333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:51.168694 3333 hybrid_clock.cc:648] HybridClock initialized: now 1786142811168601 us; error 92 us; skew 500 ppm
I20260807 22:46:51.169098 3333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:51 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142811, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:51.172796 3333 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:46:51.174492 3333 webserver.cc:460] Webserver started at http://127.20.10.129:35171/ using document root <none> and password file <none>
I20260807 22:46:51.175314 3333 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:51.175432 3333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:51.175760 3333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:51.179141 3333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "9599362e89cb41e69b48461b9360e657"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "cf60f9ed4864866cf34d79999b426152"
server_key_iv: "2c6d6ff94c9e927e2c0cc5c4b3d2e75c"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.179972 3333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "9599362e89cb41e69b48461b9360e657"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "cf60f9ed4864866cf34d79999b426152"
server_key_iv: "2c6d6ff94c9e927e2c0cc5c4b3d2e75c"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.185945 3333 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:46:51.190258 3350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.191937 3333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:51.192117 3333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "9599362e89cb41e69b48461b9360e657"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "cf60f9ed4864866cf34d79999b426152"
server_key_iv: "2c6d6ff94c9e927e2c0cc5c4b3d2e75c"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.192350 3333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:51.239497 3333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:51.244941 3333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:51.245275 3333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:51.246531 3333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:51.248432 3333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:51.248520 3333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.248622 3333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:51.248767 3333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.291636 3333 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:40921
I20260807 22:46:51.291786 3478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:40921 every 8 connection(s)
I20260807 22:46:51.293095 3333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:51.301220 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3333
I20260807 22:46:51.301496 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:51.302214 20522 external_mini_cluster.cc:1428] Setting key e54ad3c7624eac46d96753b3b1684b78
Aug 07 22:46:51 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142811, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:51.318382 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:51.296243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:45617 (local address 127.20.10.190:33961)
0807 22:46:51.296848 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:51.296853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:51.297880 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 22:46:51.300170 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:51.300177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:51.300288 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:51.300429 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:51.302295 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.303406 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.305470 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.305942 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.306102 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:51.311050 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:51.311056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:51.311059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:51.311098 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:51.314034 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:51.315355 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:51.315361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:51.315364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:51.315554 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:51.316219 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:51.316224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:51.316226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:51.316575 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:51.316726 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:51.317850 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0807 22:46:51.318073 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":282,"threads_started":1}
I20260807 22:46:51.319741 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:51.296629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:45617)
0807 22:46:51.297672 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:51.297718 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:51.299829 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:51.300714 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:51.300724 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:51.301322 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:51.302077 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:51.302097 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.303618 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.303623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:51.305276 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:51.305292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.306231 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.306240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:51.306523 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:51.307400 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:51.307428 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:51.310705 (+ 3277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:51.314255 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:51.314269 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:51.314295 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:51.315170 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:51.315705 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:51.315714 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:51.315722 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:51.316063 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:51.316749 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:51.316769 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:51.317342 (+ 573us) client_negotiation.cc:769] Sending connection context
0807 22:46:51.317700 (+ 358us) client_negotiation.cc:240] Negotiation successful
0807 22:46:51.318448 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":481,"threads_started":1}
I20260807 22:46:51.321693 3481 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:51.322139 3481 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:51.323315 3481 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:51.326211 3263 ts_manager.cc:194] Registered new tserver with Master: 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921)
I20260807 22:46:51.327873 3263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:45617
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:46:51.383391 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:51.758805 3487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:51.759135 3487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:51.759298 3487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:51.767297 3487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:51.767423 3487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:51.767608 3487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:46:51.778201 3487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:51.779719 3487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:51.781129 3487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:51.795780 3495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:51.795710 3497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:51.795831 3493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:51.797199 3487 server_base.cc:1034] running on GCE node
I20260807 22:46:51.798090 3487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:51.799327 3487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:51.800580 3487 hybrid_clock.cc:648] HybridClock initialized: now 1786142811800534 us; error 35 us; skew 500 ppm
I20260807 22:46:51.800987 3487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:51 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142811, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:51.804819 3487 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:46:51.806607 3487 webserver.cc:460] Webserver started at http://127.20.10.130:35185/ using document root <none> and password file <none>
I20260807 22:46:51.807407 3487 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:51.807546 3487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:51.807862 3487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:51.811138 3487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "157da2afcff64e15b4f79ed82acea0c0"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "d2b9ea77335cc37a40e6cc6914f8ff4a"
server_key_iv: "0c8c845af12f77f2490071d562c94ac6"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.811942 3487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "157da2afcff64e15b4f79ed82acea0c0"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "d2b9ea77335cc37a40e6cc6914f8ff4a"
server_key_iv: "0c8c845af12f77f2490071d562c94ac6"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.817567 3487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:46:51.821555 3505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.823127 3487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:51.823333 3487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "157da2afcff64e15b4f79ed82acea0c0"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "d2b9ea77335cc37a40e6cc6914f8ff4a"
server_key_iv: "0c8c845af12f77f2490071d562c94ac6"
server_key_version: "encryptionkey@0"
I20260807 22:46:51.823571 3487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:51.845877 3487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:51.851351 3487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:51.851716 3487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:51.852919 3487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:51.854568 3487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:51.854718 3487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.854926 3487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:51.854987 3487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:51.897257 3487 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44625
I20260807 22:46:51.897424 3624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44625 every 8 connection(s)
I20260807 22:46:51.898988 3487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:51.909443 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3487
I20260807 22:46:51.909669 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:51.910210 20522 external_mini_cluster.cc:1428] Setting key f893c05d1976e9506acce6433ed2d560
Aug 07 22:46:51 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142811, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:51.924249 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:51.902093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:45485 (local address 127.20.10.190:33961)
0807 22:46:51.902627 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:51.902631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:51.903851 (+ 1220us) server_negotiation.cc:407] Connection header received
0807 22:46:51.906188 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:51.906195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:51.906315 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:51.906456 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:51.908452 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.909316 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.911667 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.912418 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.912537 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:51.917306 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:51.917313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:51.917317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:51.917363 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:51.920638 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:51.921586 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:51.921591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:51.921593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:51.921739 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:51.922242 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:51.922251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:51.922253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:51.922549 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:51.922756 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:51.923560 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 22:46:51.923842 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":288,"threads_started":1}
I20260807 22:46:51.925443 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:51.902775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:45485)
0807 22:46:51.903628 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:51.903676 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:51.905832 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:51.906836 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:51.906847 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:51.907523 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:51.908226 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:51.908247 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.909550 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.909557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:51.911487 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:51.911508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:51.912810 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:51.912823 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:51.913124 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:51.914306 (+ 1182us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:51.914343 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:51.916945 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:51.920849 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:51.920867 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:51.920882 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:51.921413 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:51.921876 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:51.921881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:51.921886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:51.922101 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:51.922725 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:51.922734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:51.923133 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 22:46:51.923437 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 22:46:51.924091 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":326,"threads_started":1}
I20260807 22:46:51.927393 3626 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:51.927878 3626 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:51.928954 3626 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:51.930823 3263 ts_manager.cc:194] Registered new tserver with Master: 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625)
I20260807 22:46:51.931784 3263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:45485
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:46:51.987522 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:52.332646 3481 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
W20260807 22:46:52.392807 3632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:52.393191 3632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:52.393378 3632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:52.402347 3632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:52.402551 3632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:52.402808 3632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:46:52.415283 3632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:52.416865 3632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:52.418419 3632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:52.434854 3639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:52.434875 3638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:52.434942 3641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:52.436475 3632 server_base.cc:1034] running on GCE node
I20260807 22:46:52.437250 3632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:52.438544 3632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:52.439991 3632 hybrid_clock.cc:648] HybridClock initialized: now 1786142812439896 us; error 95 us; skew 500 ppm
I20260807 22:46:52.440445 3632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:52 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142812, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:52.444975 3632 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:46:52.447045 3632 webserver.cc:460] Webserver started at http://127.20.10.131:35057/ using document root <none> and password file <none>
I20260807 22:46:52.447806 3632 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:52.447930 3632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:52.448215 3632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:52.451653 3632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "adfe291ece1f4e2ebd04064201e132aa"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-csg5"
server_key: "c8322db236966b817e692f91d1a8caed"
server_key_iv: "8acdee17b2a663217ed12b4dad57ec2d"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.452872 3632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "adfe291ece1f4e2ebd04064201e132aa"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-csg5"
server_key: "c8322db236966b817e692f91d1a8caed"
server_key_iv: "8acdee17b2a663217ed12b4dad57ec2d"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.460012 3632 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 22:46:52.464936 3648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:52.466967 3632 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:46:52.467165 3632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "adfe291ece1f4e2ebd04064201e132aa"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-csg5"
server_key: "c8322db236966b817e692f91d1a8caed"
server_key_iv: "8acdee17b2a663217ed12b4dad57ec2d"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.467386 3632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:52.492290 3632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:52.497506 3632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:52.498092 3632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:52.499478 3632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:52.501381 3632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:52.501467 3632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:52.501660 3632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:52.501725 3632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:52.545495 3632 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33619
I20260807 22:46:52.545523 3765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33619 every 8 connection(s)
I20260807 22:46:52.547111 3632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:52.553483 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3632
I20260807 22:46:52.553704 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:52.554265 20522 external_mini_cluster.cc:1428] Setting key e21807981cbc41ab544305bbfb82e0c7
Aug 07 22:46:52 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142812, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:52.572160 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.550379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:46869 (local address 127.20.10.190:33961)
0807 22:46:52.551121 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.551131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.552042 (+ 911us) server_negotiation.cc:407] Connection header received
0807 22:46:52.554263 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.554269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.554377 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.554516 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:52.556760 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.557842 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.559710 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.560196 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.560319 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.564727 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:52.564733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:52.564736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:52.564775 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:52.568259 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.569195 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.569202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.569206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.569393 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.569960 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.569970 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.569973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.570268 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:52.570458 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.571473 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.571746 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":332,"threads_started":1}
I20260807 22:46:52.573254 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.550835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:46869)
0807 22:46:52.551855 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.551900 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.553938 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.554925 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.554937 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.555682 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:52.556528 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.556558 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.558020 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.558025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.559554 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.559569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.560462 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.560467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.560829 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.561631 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:52.561657 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:52.564409 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:52.568477 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:52.568486 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:52.568503 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:52.569058 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:52.569546 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:52.569550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:52.569553 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:52.569818 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:52.570438 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:52.570447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:52.570998 (+ 551us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.571342 (+ 344us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.571931 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":513,"threads_started":1}
I20260807 22:46:52.574985 3766 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:52.575346 3766 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:52.576206 3766 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:52.577870 3263 ts_manager.cc:194] Registered new tserver with Master: adfe291ece1f4e2ebd04064201e132aa (127.20.10.131:33619)
I20260807 22:46:52.579012 3263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:46869
I20260807 22:46:52.584713 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:52.605567 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.590326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.20.10.190:33961)
0807 22:46:52.590613 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.590621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.591023 (+ 402us) server_negotiation.cc:407] Connection header received
0807 22:46:52.591256 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.591261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.591368 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.591457 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:52.593003 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.593803 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.595358 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.595803 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.595958 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.599036 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:52.599041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:52.599045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:52.599083 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:52.602065 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.602995 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.602999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.603002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.603238 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.603691 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.603697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.603700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.604099 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:52.604419 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.604776 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.604998 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 22:46:52.616194 3263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:52.623759 3263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:52.663924 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.638736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33619 (local address 127.0.0.1:56668)
0807 22:46:52.650640 (+ 11904us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.650756 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.651057 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.654076 (+ 3019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.654086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.654124 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.654725 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.654740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.657333 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.657338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.660360 (+ 3022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.660382 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.662724 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.662732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.662973 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.663060 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.663191 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.663426 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11480,"mutex_wait_us":68,"thread_start_us":14257,"threads_started":1}
I20260807 22:46:52.664223 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.639710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.20.10.129:40921)
0807 22:46:52.653455 (+ 13745us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.653462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.653489 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:46:52.653721 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.653733 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.653965 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.654163 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.655322 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.657206 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.661378 (+ 4172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.662953 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.663105 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.663350 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.663674 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.663863 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13576,"thread_start_us":13403,"threads_started":1}
I20260807 22:46:52.664873 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.639069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.20.10.131:33619)
0807 22:46:52.653078 (+ 14009us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.653099 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.653143 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:46:52.653361 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.653371 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.653785 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.654960 (+ 1175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.655334 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.657123 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.660557 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.662124 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.662262 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.662537 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.664416 (+ 1879us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.664607 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13800,"thread_start_us":13603,"threads_started":1}
I20260807 22:46:52.664860 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.639345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58606)
0807 22:46:52.652648 (+ 13303us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.652679 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.652874 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.654462 (+ 1588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.654468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.654509 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.655074 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.655089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.657384 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.657391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.661217 (+ 3826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.661230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.663205 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.663211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.663430 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.663474 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.663585 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.663761 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13088,"mutex_wait_us":1217,"thread_start_us":11915,"threads_started":1}
I20260807 22:46:52.669232 3407 tablet_service.cc:1467] Processing CreateTablet for tablet 0def4b7fa2414acca6e2203154c190ce (DEFAULT_TABLE table=test-table [id=c1baa6fceca442138bf86102e3b33ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:52.669246 3696 tablet_service.cc:1467] Processing CreateTablet for tablet 0def4b7fa2414acca6e2203154c190ce (DEFAULT_TABLE table=test-table [id=c1baa6fceca442138bf86102e3b33ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:52.670073 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.653777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:44625 (local address 127.0.0.1:50626)
0807 22:46:52.659172 (+ 5395us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.659194 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.659420 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.660160 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.660165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.660195 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.660654 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.660664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.664424 (+ 3760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.664431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.666855 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.666879 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.669222 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.669227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.669379 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.669404 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.669524 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.669675 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5218,"thread_start_us":5011,"threads_started":1}
I20260807 22:46:52.670337 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.653763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.20.10.130:44625)
0807 22:46:52.658489 (+ 4726us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.658499 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.659300 (+ 801us) server_negotiation.cc:407] Connection header received
0807 22:46:52.659642 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.659651 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.659964 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.660168 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.661410 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.663512 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.667086 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.668798 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.668903 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.669053 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.669587 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.669766 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4520,"thread_start_us":4391,"threads_started":1}
I20260807 22:46:52.671003 3696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0def4b7fa2414acca6e2203154c190ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:52.671070 3407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0def4b7fa2414acca6e2203154c190ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:52.674398 3557 tablet_service.cc:1467] Processing CreateTablet for tablet 0def4b7fa2414acca6e2203154c190ce (DEFAULT_TABLE table=test-table [id=c1baa6fceca442138bf86102e3b33ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:52.675886 3557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0def4b7fa2414acca6e2203154c190ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:52.688529 3788 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Bootstrap starting.
I20260807 22:46:52.689085 3787 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Bootstrap starting.
I20260807 22:46:52.693463 3788 tablet_bootstrap.cc:654] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:52.693964 3787 tablet_bootstrap.cc:654] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:52.694936 3788 log.cc:824] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:52.695827 3787 log.cc:824] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:52.696867 3789 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Bootstrap starting.
I20260807 22:46:52.699285 3788 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: No bootstrap required, opened a new log
I20260807 22:46:52.699687 3788 ts_tablet_manager.cc:1397] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260807 22:46:52.701798 3787 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: No bootstrap required, opened a new log
I20260807 22:46:52.701846 3789 tablet_bootstrap.cc:654] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:52.702143 3787 ts_tablet_manager.cc:1397] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 22:46:52.703439 3789 log.cc:824] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:52.707477 3789 tablet_bootstrap.cc:492] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: No bootstrap required, opened a new log
I20260807 22:46:52.707845 3789 ts_tablet_manager.cc:1397] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 22:46:52.711131 3787 raft_consensus.cc:348] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.711742 3787 raft_consensus.cc:374] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:52.711874 3787 raft_consensus.cc:729] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9599362e89cb41e69b48461b9360e657, State: Initialized, Role: FOLLOWER
I20260807 22:46:52.712770 3787 consensus_queue.cc:260] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.712899 3788 raft_consensus.cc:348] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.713436 3788 raft_consensus.cc:374] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:52.713554 3788 raft_consensus.cc:729] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfe291ece1f4e2ebd04064201e132aa, State: Initialized, Role: FOLLOWER
I20260807 22:46:52.715044 3788 consensus_queue.cc:260] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.715373 3787 ts_tablet_manager.cc:1428] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260807 22:46:52.716764 3766 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
I20260807 22:46:52.717541 3788 ts_tablet_manager.cc:1428] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Time spent starting tablet: real 0.018s user 0.016s sys 0.001s
I20260807 22:46:52.720659 3789 raft_consensus.cc:348] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.721292 3789 raft_consensus.cc:374] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:52.721415 3789 raft_consensus.cc:729] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157da2afcff64e15b4f79ed82acea0c0, State: Initialized, Role: FOLLOWER
I20260807 22:46:52.722231 3789 consensus_queue.cc:260] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.723538 3626 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
I20260807 22:46:52.724159 3789 ts_tablet_manager.cc:1428] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260807 22:46:52.796386 3482 tablet.cc:2369] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:46:52.798995 3767 tablet.cc:2369] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:52.872963 3794 raft_consensus.cc:482] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:52.873327 3794 raft_consensus.cc:504] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.875131 3795 raft_consensus.cc:482] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:52.875538 3794 leader_election.cc:288] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921), 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625)
I20260807 22:46:52.875483 3795 raft_consensus.cc:504] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.884163 3795 leader_election.cc:288] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfe291ece1f4e2ebd04064201e132aa (127.20.10.131:33619), 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921)
I20260807 22:46:52.884898 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.875783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.20.10.131:39793)
0807 22:46:52.876025 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.876052 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.876204 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.877296 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.877302 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.877340 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.877639 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.877653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.879356 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.879362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.881739 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.881759 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.883907 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.883914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.884140 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.884188 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.884317 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.884500 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"spinlock_wait_cycles":24960}
I20260807 22:46:52.885634 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.876422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:39793 (local address 127.20.10.129:40921)
0807 22:46:52.876652 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.876661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.876682 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:46:52.876788 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.876796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.876900 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.877010 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.877832 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.879156 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.881997 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.883503 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.883702 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.883800 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.885166 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.885349 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:46:52.885666 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.876480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:50741 (local address 127.20.10.130:44625)
0807 22:46:52.876704 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.876710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.877392 (+ 682us) server_negotiation.cc:407] Connection header received
0807 22:46:52.877622 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.877626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.877689 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.877809 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.878631 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.879876 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.882763 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.884270 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.884409 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.884523 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.885225 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.885374 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:46:52.887238 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.876094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:44625 (local address 127.20.10.131:50741)
0807 22:46:52.877273 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.877289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.877465 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.878014 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.878018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.878044 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.878436 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.878449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.880117 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.880123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.882490 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.882509 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.884738 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.884745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.884965 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.885013 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.885874 (+ 861us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.886825 (+ 951us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":793,"threads_started":1}
I20260807 22:46:52.887538 3428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9599362e89cb41e69b48461b9360e657" is_pre_election: true
I20260807 22:46:52.888334 3428 raft_consensus.cc:2454] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 0.
I20260807 22:46:52.889140 3577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157da2afcff64e15b4f79ed82acea0c0" is_pre_election: true
I20260807 22:46:52.889479 3649 leader_election.cc:302] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9599362e89cb41e69b48461b9360e657, adfe291ece1f4e2ebd04064201e132aa; no voters:
I20260807 22:46:52.889882 3577 raft_consensus.cc:2454] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 0.
I20260807 22:46:52.890029 3794 raft_consensus.cc:2781] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:52.890213 3794 raft_consensus.cc:482] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:52.890324 3794 raft_consensus.cc:3037] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:52.894289 3794 raft_consensus.cc:504] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.895792 3577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157da2afcff64e15b4f79ed82acea0c0"
I20260807 22:46:52.895785 3428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9599362e89cb41e69b48461b9360e657"
I20260807 22:46:52.896085 3577 raft_consensus.cc:3037] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:52.896072 3428 raft_consensus.cc:3037] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:52.897914 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.878866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:57387 (local address 127.20.10.129:40921)
0807 22:46:52.886086 (+ 7220us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.886092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.888438 (+ 2346us) server_negotiation.cc:407] Connection header received
0807 22:46:52.888772 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.888777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.888867 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.888931 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.889985 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.891338 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.894457 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.896045 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.896292 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.896385 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.897252 (+ 867us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.897445 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7091,"thread_start_us":7505,"threads_started":1}
I20260807 22:46:52.898948 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.878193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.20.10.130:57387)
0807 22:46:52.888308 (+ 10115us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.888336 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.888581 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.889197 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.889203 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.889235 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.889779 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.889795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.891547 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.891553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.894219 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.894244 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.896684 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.896693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.897003 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.897040 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.898186 (+ 1146us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.898383 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9927,"thread_start_us":9662,"threads_started":1}
I20260807 22:46:52.900179 3428 raft_consensus.cc:2454] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 1.
I20260807 22:46:52.900830 3649 leader_election.cc:302] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9599362e89cb41e69b48461b9360e657, adfe291ece1f4e2ebd04064201e132aa; no voters:
W20260807 22:46:52.901955 3627 tablet.cc:2369] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:52.902485 3577 raft_consensus.cc:2454] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 1.
I20260807 22:46:52.903163 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.880180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:60587 (local address 127.20.10.131:33619)
0807 22:46:52.880480 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.880490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.891804 (+ 11314us) server_negotiation.cc:407] Connection header received
0807 22:46:52.892089 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.892095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.892189 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.892306 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:52.893048 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.893897 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.899951 (+ 6054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.901477 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.901658 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.901766 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.902496 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.902698 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:46:52.903555 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.878304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33619 (local address 127.20.10.130:60587)
0807 22:46:52.891690 (+ 13386us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:52.891716 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:52.891923 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:52.892339 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:52.892345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:52.892379 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:52.892811 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.892821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.894058 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.894064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.899693 (+ 5629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:52.899713 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.902022 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.902030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:52.902256 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.902294 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:46:52.902861 (+ 567us) client_negotiation.cc:240] Negotiation successful
0807 22:46:52.903098 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13219,"thread_start_us":12987,"threads_started":1}
I20260807 22:46:52.904455 3800 raft_consensus.cc:2781] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:52.902804 3794 leader_election.cc:288] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 election: Requested vote from peers 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921), 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625)
I20260807 22:46:52.904838 3717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "157da2afcff64e15b4f79ed82acea0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfe291ece1f4e2ebd04064201e132aa" is_pre_election: true
I20260807 22:46:52.905884 3428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0def4b7fa2414acca6e2203154c190ce" candidate_uuid: "157da2afcff64e15b4f79ed82acea0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9599362e89cb41e69b48461b9360e657" is_pre_election: true
I20260807 22:46:52.906291 3428 raft_consensus.cc:2379] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 157da2afcff64e15b4f79ed82acea0c0 in current term 1: Already voted for candidate adfe291ece1f4e2ebd04064201e132aa in this term.
I20260807 22:46:52.907138 3800 raft_consensus.cc:686] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 1 LEADER]: Becoming Leader. State: Replica: adfe291ece1f4e2ebd04064201e132aa, State: Running, Role: LEADER
I20260807 22:46:52.907789 3800 consensus_queue.cc:237] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } }
I20260807 22:46:52.912971 3509 leader_election.cc:302] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 157da2afcff64e15b4f79ed82acea0c0; no voters: 9599362e89cb41e69b48461b9360e657, adfe291ece1f4e2ebd04064201e132aa
I20260807 22:46:52.913512 3795 raft_consensus.cc:2726] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 22:46:52.914220 3263 catalog_manager.cc:5495] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa reported cstate change: term changed from 0 to 1, leader changed from <none> to adfe291ece1f4e2ebd04064201e132aa (127.20.10.131). New cstate: current_term: 1 leader_uuid: "adfe291ece1f4e2ebd04064201e132aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } health_report { overall_health: UNKNOWN } } }
I20260807 22:46:52.977541 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.970264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.20.10.131:33619)
0807 22:46:52.970471 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.970476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.970601 (+ 125us) server_negotiation.cc:407] Connection header received
0807 22:46:52.970923 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.970928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.971040 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.971191 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:52.971882 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.972896 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.974589 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.975230 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.975366 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.976089 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:52.976764 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:52.976893 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.977106 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.977190 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:46:52.987371 3428 raft_consensus.cc:1261] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Refusing update from remote peer adfe291ece1f4e2ebd04064201e132aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:52.987434 3577 raft_consensus.cc:1261] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Refusing update from remote peer adfe291ece1f4e2ebd04064201e132aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:52.988503 3801 consensus_queue.cc:1035] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:52.988960 3800 consensus_queue.cc:1035] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:53.002432 3804 mvcc.cc:205] Tried to move back new op lower bound from 7316040961984012288 to 7316040961674850304. Current Snapshot: MvccSnapshot[applied={T|T < 7316040961984012288}]
I20260807 22:46:53.022979 3262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:53.023355 3262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:53.028451 3262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Sending DeleteTablet for 3 replicas of tablet 0def4b7fa2414acca6e2203154c190ce
I20260807 22:46:53.030014 3407 tablet_service.cc:1514] Processing DeleteTablet for tablet 0def4b7fa2414acca6e2203154c190ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:53 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:58606
I20260807 22:46:53.030018 3696 tablet_service.cc:1514] Processing DeleteTablet for tablet 0def4b7fa2414acca6e2203154c190ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:53 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56668
I20260807 22:46:53.030913 3557 tablet_service.cc:1514] Processing DeleteTablet for tablet 0def4b7fa2414acca6e2203154c190ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:53 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:50626
I20260807 22:46:53.032271 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3333
I20260807 22:46:53.040160 3815 tablet_replica.cc:330] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: stopping tablet replica
I20260807 22:46:53.040284 3814 tablet_replica.cc:330] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: stopping tablet replica
I20260807 22:46:53.040858 3814 raft_consensus.cc:2229] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:53.041169 3815 raft_consensus.cc:2229] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:53.041742 3814 raft_consensus.cc:2258] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:53.041986 3815 raft_consensus.cc:2258] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:53.042641 3813 tablet_replica.cc:330] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: stopping tablet replica
I20260807 22:46:53.043195 3813 raft_consensus.cc:2229] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:53.043778 3813 raft_consensus.cc:2258] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:53.046131 3814 ts_tablet_manager.cc:1905] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:53.047991 3813 ts_tablet_manager.cc:1905] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:53.053953 3813 ts_tablet_manager.cc:1918] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:53.054210 3813 log.cc:1196] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/0def4b7fa2414acca6e2203154c190ce
I20260807 22:46:53.054287 3815 ts_tablet_manager.cc:1905] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:53.055032 3813 ts_tablet_manager.cc:1939] T 0def4b7fa2414acca6e2203154c190ce P 9599362e89cb41e69b48461b9360e657: Deleting consensus metadata
I20260807 22:46:53.057729 3246 catalog_manager.cc:4841] TS 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921): tablet 0def4b7fa2414acca6e2203154c190ce (table test-table [id=c1baa6fceca442138bf86102e3b33ea0]) successfully deleted
I20260807 22:46:53.055720 3814 ts_tablet_manager.cc:1918] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:53.058694 3814 log.cc:1196] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/0def4b7fa2414acca6e2203154c190ce
I20260807 22:46:53.059628 3814 ts_tablet_manager.cc:1939] T 0def4b7fa2414acca6e2203154c190ce P adfe291ece1f4e2ebd04064201e132aa: Deleting consensus metadata
I20260807 22:46:53.061719 3249 catalog_manager.cc:4841] TS adfe291ece1f4e2ebd04064201e132aa (127.20.10.131:33619): tablet 0def4b7fa2414acca6e2203154c190ce (table test-table [id=c1baa6fceca442138bf86102e3b33ea0]) successfully deleted
I20260807 22:46:53.062974 3815 ts_tablet_manager.cc:1918] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:53.063297 3815 log.cc:1196] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/0def4b7fa2414acca6e2203154c190ce
I20260807 22:46:53.063959 3815 ts_tablet_manager.cc:1939] T 0def4b7fa2414acca6e2203154c190ce P 157da2afcff64e15b4f79ed82acea0c0: Deleting consensus metadata
I20260807 22:46:53.065874 3249 catalog_manager.cc:4841] TS 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625): tablet 0def4b7fa2414acca6e2203154c190ce (table test-table [id=c1baa6fceca442138bf86102e3b33ea0]) successfully deleted
Aug 07 22:46:53 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:46:53.080220 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:53.034128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.20.10.129:40921)
0807 22:46:53.034411 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:53.034422 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:53.047156 (+ 12734us) server_negotiation.cc:407] Connection header received
0807 22:46:53.047383 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:53.047389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:53.047525 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:53.047645 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:53.058016 (+ 10371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.059225 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.062174 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.062949 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.063095 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:53.071383 (+ 8288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:53.071408 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:53.071435 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:53.071499 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:53.076116 (+ 4617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.077398 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.077411 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.077414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.077627 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.078163 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.078172 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.078178 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.078777 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:53.079161 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:53.079510 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 22:46:53.079807 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 22:46:53.081085 3474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:53.183105 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3333
I20260807 22:46:53.216590 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3487
Aug 07 22:46:53 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:46:53.238260 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:53.217541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.20.10.130:44625)
0807 22:46:53.217821 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:53.217826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:53.217846 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:46:53.218063 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:53.218070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:53.218310 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:53.218424 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:53.220329 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.221106 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.223232 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.223683 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.223899 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:53.230631 (+ 6732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:53.230696 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:53.230721 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:53.230792 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:53.234242 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.235313 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.235322 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.235324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.235504 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.236008 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.236012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.236015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.236471 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:53.236845 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:53.237231 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 22:46:53.237513 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260807 22:46:53.239071 3619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:53.345184 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3487
I20260807 22:46:53.370596 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3632
Aug 07 22:46:53 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:46:53.388923 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:53.371597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.20.10.131:33619)
0807 22:46:53.371823 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:53.371829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:53.371852 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:46:53.372065 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:53.372075 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:53.372183 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:53.372331 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:53.373816 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.374578 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.376013 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.376479 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.376654 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:53.381504 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:53.381519 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:53.381535 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:53.381581 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:53.385050 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.385959 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.385968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.385974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.386113 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:53.386744 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:53.386750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:53.386753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:53.387278 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:53.387695 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:53.388276 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 22:46:53.388553 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:46:53.389758 3761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:53.528754 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3632
I20260807 22:46:53.557314 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3229
I20260807 22:46:53.558898 3295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:53.677364 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3229
I20260807 22:46:53.707504 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--webserver_interface=127.20.10.190
--webserver_port=33585
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:54.074795 3821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:54.075163 3821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:54.075351 3821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:54.083384 3821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:54.083513 3821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:54.083611 3821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:54.083720 3821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:54.083806 3821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:54.094954 3821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=33585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:54.096446 3821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:54.097852 3821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:54.130350 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:54.130530 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:54.130236 3830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:54.131515 3821 server_base.cc:1034] running on GCE node
I20260807 22:46:54.132623 3821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:54.134538 3821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:54.135841 3821 hybrid_clock.cc:648] HybridClock initialized: now 1786142814135778 us; error 105 us; skew 500 ppm
I20260807 22:46:54.136340 3821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:54 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142814, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:54.140883 3821 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:54.143131 3821 webserver.cc:460] Webserver started at http://127.20.10.190:33585/ using document root <none> and password file <none>
I20260807 22:46:54.143988 3821 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:54.144131 3821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:54.151423 3821 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:46:54.155361 3837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:54.157168 3821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:54.157395 3821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "aa49b0a062de43059eeb3d7769ffa141"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-csg5"
server_key: "652a4b21e954ef90783332d8d702b420"
server_key_iv: "defdf3afc97a355d5d8e030cfb8da2c7"
server_key_version: "encryptionkey@0"
I20260807 22:46:54.158752 3821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:54.178943 3821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:54.184266 3821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:54.184732 3821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:54.207705 3821 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33961
I20260807 22:46:54.207718 3891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33961 every 8 connection(s)
I20260807 22:46:54.209434 3821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:54.213042 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3821
I20260807 22:46:54.214264 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:40921
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35171
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:54.221117 3892 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:46:54.227768 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap starting.
I20260807 22:46:54.254076 3892 log.cc:824] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:54.268272 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:46:54.268930 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap complete.
I20260807 22:46:54.279537 3892 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:54.280475 3892 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Initialized, Role: FOLLOWER
I20260807 22:46:54.281294 3892 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:54.281599 3892 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:54.281814 3892 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:54.282033 3892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:46:54.286589 3892 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:54.287458 3892 leader_election.cc:302] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa49b0a062de43059eeb3d7769ffa141; no voters:
I20260807 22:46:54.288189 3892 leader_election.cc:288] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:46:54.288358 3896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:46:54.289862 3896 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 LEADER]: Becoming Leader. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Running, Role: LEADER
I20260807 22:46:54.290612 3896 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:54.290935 3892 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:54.293816 3898 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa49b0a062de43059eeb3d7769ffa141. Latest consensus state: current_term: 2 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:54.294319 3898 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:54.295015 3897 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:54.295401 3897 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:54.307860 3903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:54.317922 3903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:54.319947 3903 catalog_manager.cc:1259] Loaded cluster ID: 0c380c4d449944c6a4d7d2652a858b19
I20260807 22:46:54.320200 3903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:54.326530 3903 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 22:46:54.339156 3903 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 22:46:54.347257 3903 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:54.349217 3903 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Loaded TSK: 0
I20260807 22:46:54.351182 3903 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:46:54.667589 3894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:54.667985 3894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:54.668159 3894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:54.676936 3894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:54.677168 3894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:54.677367 3894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:46:54.690347 3894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:40921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=35171
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:54.692082 3894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:54.693691 3894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:54.712608 3920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:54.712958 3922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:54.712700 3919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:54.714558 3894 server_base.cc:1034] running on GCE node
I20260807 22:46:54.715373 3894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:54.716588 3894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:54.717869 3894 hybrid_clock.cc:648] HybridClock initialized: now 1786142814717782 us; error 79 us; skew 500 ppm
I20260807 22:46:54.718266 3894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:54 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142814, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:54.722122 3894 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:46:54.724272 3894 webserver.cc:460] Webserver started at http://127.20.10.129:35171/ using document root <none> and password file <none>
I20260807 22:46:54.725128 3894 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:54.725296 3894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:54.731640 3894 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:46:54.735015 3929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:54.736752 3894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:46:54.736945 3894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "9599362e89cb41e69b48461b9360e657"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "cf60f9ed4864866cf34d79999b426152"
server_key_iv: "2c6d6ff94c9e927e2c0cc5c4b3d2e75c"
server_key_version: "encryptionkey@0"
I20260807 22:46:54.738080 3894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:54.751924 3894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:54.757123 3894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:54.757388 3894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:54.758558 3894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:54.760185 3894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:54.760285 3894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:54.760428 3894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:54.760524 3894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:54.811344 3894 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:40921
I20260807 22:46:54.811403 4050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:40921 every 8 connection(s)
I20260807 22:46:54.813136 3894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:54.814172 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 3894
I20260807 22:46:54.815032 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:44625
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35185
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:54 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142814, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:54.843482 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:54.816624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:45369 (local address 127.20.10.190:33961)
0807 22:46:54.817329 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:54.817348 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:54.818695 (+ 1347us) server_negotiation.cc:407] Connection header received
0807 22:46:54.821458 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:54.821478 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:54.822020 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:54.822447 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:54.824267 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:54.826359 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:54.828691 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:54.829559 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:54.829735 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:54.834828 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:54.834839 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:54.834863 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:54.834908 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:54.838146 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:54.839179 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:54.839187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:54.839192 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:54.839397 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:54.839902 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:54.839914 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:54.839917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:54.840496 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:54.840722 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:54.841569 (+ 847us) server_negotiation.cc:299] Negotiation successful
0807 22:46:54.841983 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":303,"threads_started":1}
I20260807 22:46:54.844444 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:54.816849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:45369)
0807 22:46:54.818424 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:54.818487 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:54.820716 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:54.822636 (+ 1920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:54.822649 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:54.823385 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:54.824004 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:54.824023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:54.826591 (+ 2568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:54.826596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:54.828480 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:54.828498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:54.829852 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:54.829863 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:54.830254 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:54.831163 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:54.831204 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:54.834415 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:54.838351 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:54.838361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:54.838378 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:54.838998 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:54.839543 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:54.839548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:54.839553 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:54.839766 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:54.840659 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:54.840668 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:54.841034 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 22:46:54.841327 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 22:46:54.841963 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"thread_start_us":950,"threads_started":1}
I20260807 22:46:54.847980 4051 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:54.848400 4051 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:54.849252 4051 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:54.851864 3855 ts_manager.cc:194] Registered new tserver with Master: 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921)
I20260807 22:46:54.853479 3855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:45369
W20260807 22:46:55.222903 4055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:55.223232 4055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:55.223446 4055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:55.233701 4055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:55.233894 4055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:55.234205 4055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:46:55.248145 4055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:44625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=35185
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:55.249523 4055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:55.251006 4055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:55.266403 4061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.266547 4064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.266508 4062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:55.267679 4055 server_base.cc:1034] running on GCE node
I20260807 22:46:55.268358 4055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:55.269441 4055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:55.270773 4055 hybrid_clock.cc:648] HybridClock initialized: now 1786142815270688 us; error 64 us; skew 500 ppm
I20260807 22:46:55.271222 4055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:55 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:55.274948 4055 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:46:55.276835 4055 webserver.cc:460] Webserver started at http://127.20.10.130:35185/ using document root <none> and password file <none>
I20260807 22:46:55.277709 4055 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:55.277907 4055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:55.284657 4055 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 22:46:55.288201 4071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.290105 4055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:46:55.290336 4055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "157da2afcff64e15b4f79ed82acea0c0"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "d2b9ea77335cc37a40e6cc6914f8ff4a"
server_key_iv: "0c8c845af12f77f2490071d562c94ac6"
server_key_version: "encryptionkey@0"
I20260807 22:46:55.291676 4055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:55.319096 4055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:55.324370 4055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:55.324707 4055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:55.326020 4055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:55.327852 4055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:55.327994 4055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.328142 4055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:55.328239 4055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.368834 4055 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44625
I20260807 22:46:55.368990 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44625 every 8 connection(s)
I20260807 22:46:55.370487 4055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:55.374326 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4055
I20260807 22:46:55.375233 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:33619
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35057
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:46:55 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:55.397095 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.373581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:56443 (local address 127.20.10.190:33961)
0807 22:46:55.374183 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.374188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.375324 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 22:46:55.377589 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.377596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.377719 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.377863 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:55.379780 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.380516 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.382846 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.383450 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.383650 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.388796 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:55.388803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:55.388806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:55.388853 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:55.392463 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.393750 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.393757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.393760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.393942 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.394595 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.394599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.394601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.395028 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:55.395161 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.396414 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.396724 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":361,"threads_started":1}
I20260807 22:46:55.398543 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.374003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:56443)
0807 22:46:55.375126 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:55.375174 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:55.377214 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:55.378160 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:55.378171 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:55.378858 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:55.379536 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.379557 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.380722 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.380727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.382553 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.382573 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.383793 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.383804 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.384125 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.385013 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:55.385042 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:55.388345 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:55.392690 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:55.392713 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:55.392740 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:55.393565 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:55.394081 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:55.394088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:55.394094 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:55.394445 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:55.395183 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:55.395195 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:55.395760 (+ 565us) client_negotiation.cc:769] Sending connection context
0807 22:46:55.396102 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 22:46:55.396792 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":560,"threads_started":1}
I20260807 22:46:55.400626 4192 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:55.401108 4192 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:55.402063 4192 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:55.404095 3855 ts_manager.cc:194] Registered new tserver with Master: 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625)
I20260807 22:46:55.405138 3855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:56443
W20260807 22:46:55.749722 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:55.750131 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:55.750329 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:55.758113 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:55.758315 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:55.758517 4197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:46:55.769573 4197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:33619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=35057
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:55.771209 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:55.772744 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:55.787521 4205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.787899 4204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.787632 4207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:55.788905 4197 server_base.cc:1034] running on GCE node
I20260807 22:46:55.789675 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:55.790901 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:55.792160 4197 hybrid_clock.cc:648] HybridClock initialized: now 1786142815792087 us; error 61 us; skew 500 ppm
I20260807 22:46:55.792603 4197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:55 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:55.796173 4197 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:46:55.797885 4197 webserver.cc:460] Webserver started at http://127.20.10.131:35057/ using document root <none> and password file <none>
I20260807 22:46:55.798586 4197 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:55.798779 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:55.804910 4197 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:46:55.807974 4214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.809672 4197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:55.809894 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "adfe291ece1f4e2ebd04064201e132aa"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-csg5"
server_key: "c8322db236966b817e692f91d1a8caed"
server_key_iv: "8acdee17b2a663217ed12b4dad57ec2d"
server_key_version: "encryptionkey@0"
I20260807 22:46:55.811426 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:55.837846 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:55.842939 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:55.843268 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:55.844552 4197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:55.846177 4197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:55.846295 4197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.846434 4197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:55.846503 4197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.857084 4051 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
I20260807 22:46:55.887629 4197 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33619
I20260807 22:46:55.887835 4334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33619 every 8 connection(s)
I20260807 22:46:55.889463 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:55.895365 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4197
Aug 07 22:46:55 dist-test-slave-csg5 krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:46:55.917364 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.896595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.20.10.190:33961)
0807 22:46:55.897544 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.897548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.897565 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:46:55.897608 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.897611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.897661 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.897752 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:55.900487 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.901771 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.904094 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.904868 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.905021 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.909635 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:55.909644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:55.909649 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:55.909704 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:55.913440 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.914613 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.914621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.914623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.914878 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.915356 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.915361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.915364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.915838 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:55.916097 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.916657 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.916906 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":683,"threads_started":1}
I20260807 22:46:55.919227 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.892718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:46927 (local address 127.20.10.190:33961)
0807 22:46:55.892991 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.892997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.894340 (+ 1343us) server_negotiation.cc:407] Connection header received
0807 22:46:55.896926 (+ 2586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.896934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.897069 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.897166 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:55.900286 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.901725 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.904447 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.905199 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.905327 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.910869 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:55.910878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:55.910885 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:55.910953 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:55.914762 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.915978 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.915984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.915987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.916141 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:55.916755 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:55.916760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:55.916763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:55.917279 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:55.917408 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.918590 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.918898 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 22:46:55.920600 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.893261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:46927)
0807 22:46:55.894146 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:55.894191 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:55.896446 (+ 2255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:55.897890 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:55.897905 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:55.898889 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:55.900012 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.900047 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.902030 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.902036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.904277 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.904294 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.905543 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.905555 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.906041 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.906829 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:55.906881 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:55.910349 (+ 3468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:55.914960 (+ 4611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:55.914974 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:55.914997 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:55.915742 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:55.916275 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:55.916279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:55.916281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:55.916575 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:55.917437 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:55.917447 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:55.918120 (+ 673us) client_negotiation.cc:769] Sending connection context
0807 22:46:55.918493 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 22:46:55.919158 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":334,"threads_started":1}
I20260807 22:46:55.922266 4335 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33961
I20260807 22:46:55.922700 4335 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:55.923527 4335 heartbeater.cc:511] Master 127.20.10.190:33961 requested a full tablet report, sending...
I20260807 22:46:55.925206 3855 ts_manager.cc:194] Registered new tserver with Master: adfe291ece1f4e2ebd04064201e132aa (127.20.10.131:33619)
I20260807 22:46:55.926276 3855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:46927
I20260807 22:46:55.933964 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:55.942428 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.934984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.20.10.190:33961)
0807 22:46:55.935234 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.935242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.935732 (+ 490us) server_negotiation.cc:407] Connection header received
0807 22:46:55.935946 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.935950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.936105 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.936250 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:55.936746 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.937566 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.939746 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.940262 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.940396 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.941056 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:55.941600 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:55.941718 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.941946 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.942065 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:46:55.943651 3855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:55.951432 3855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:55.982077 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.970116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.20.10.130:44625)
0807 22:46:55.972395 (+ 2279us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.972405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.972890 (+ 485us) server_negotiation.cc:407] Connection header received
0807 22:46:55.973044 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.973054 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.973354 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.973610 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:55.975126 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.976281 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.978805 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.980263 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.980500 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.980761 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.981507 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.981700 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2118,"thread_start_us":2011,"threads_started":1}
I20260807 22:46:55.982167 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.971351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:44625 (local address 127.0.0.1:50656)
0807 22:46:55.972691 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:55.972712 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:55.972856 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:55.973913 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:55.973919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:55.973950 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:55.974854 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.974874 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.976435 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.976446 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.978588 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.978609 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.980946 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.980954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.981191 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.981258 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 22:46:55.981431 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:46:55.981701 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":47,"thread_start_us":1026,"threads_started":1}
I20260807 22:46:55.982251 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.966601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.20.10.131:33619)
0807 22:46:55.967799 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.967811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.968873 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 22:46:55.969427 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.969436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.970881 (+ 1445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.971027 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:55.972235 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.974184 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.978498 (+ 4314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.980331 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.980481 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.980823 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.981589 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.981821 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":995,"thread_start_us":762,"threads_started":1}
I20260807 22:46:55.982805 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.967863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.20.10.129:40921)
0807 22:46:55.970785 (+ 2922us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.970800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.970845 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:46:55.970982 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.970995 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.971304 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.971534 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:55.973065 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.975295 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.978970 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.981038 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.981183 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.981459 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.982175 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.982415 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2499,"thread_start_us":2290,"threads_started":1}
I20260807 22:46:55.983295 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.967127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58620)
0807 22:46:55.969666 (+ 2539us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:55.969690 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:55.969840 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:55.971541 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:55.971548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:55.971582 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:55.972110 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.972122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.975485 (+ 3363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.975492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.978419 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.978439 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.981691 (+ 3252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.981698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.981908 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.981956 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:46:55.982686 (+ 730us) client_negotiation.cc:240] Negotiation successful
0807 22:46:55.982845 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"mutex_wait_us":2,"spinlock_wait_cycles":7808,"thread_start_us":918,"threads_started":1}
I20260807 22:46:55.983494 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.966245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33619 (local address 127.0.0.1:56710)
0807 22:46:55.968625 (+ 2380us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:55.968692 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:55.969112 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:55.971349 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:55.971354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:55.971382 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:55.971947 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.971965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.974602 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.974612 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.978196 (+ 3584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:55.978227 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:55.981103 (+ 2876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:55.981110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:55.981326 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:55.981365 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:46:55.982565 (+ 1200us) client_negotiation.cc:240] Negotiation successful
0807 22:46:55.982779 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2012,"mutex_wait_us":46,"spinlock_wait_cycles":1280,"thread_start_us":3100,"threads_started":1}
I20260807 22:46:55.986740 4122 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 (DEFAULT_TABLE table=test-table [id=6f7e1ab6fa024379b524bf2d6be777bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:55.987213 3978 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 (DEFAULT_TABLE table=test-table [id=6f7e1ab6fa024379b524bf2d6be777bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:55.987213 4266 tablet_service.cc:1467] Processing CreateTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 (DEFAULT_TABLE table=test-table [id=6f7e1ab6fa024379b524bf2d6be777bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:55.988561 4122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb86d68fb04474ae1e0610e6be12f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:55.988610 4266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb86d68fb04474ae1e0610e6be12f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:55.988724 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8cb86d68fb04474ae1e0610e6be12f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:56.003680 4354 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Bootstrap starting.
I20260807 22:46:56.004859 4355 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Bootstrap starting.
I20260807 22:46:56.006264 4356 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Bootstrap starting.
I20260807 22:46:56.008472 4354 tablet_bootstrap.cc:654] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:56.009109 4355 tablet_bootstrap.cc:654] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:56.009352 4356 tablet_bootstrap.cc:654] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:56.010468 4355 log.cc:824] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:56.010728 4356 log.cc:824] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:56.010380 4354 log.cc:824] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:56.015827 4356 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: No bootstrap required, opened a new log
I20260807 22:46:56.015878 4355 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: No bootstrap required, opened a new log
I20260807 22:46:56.015878 4354 tablet_bootstrap.cc:492] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: No bootstrap required, opened a new log
I20260807 22:46:56.016187 4356 ts_tablet_manager.cc:1397] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260807 22:46:56.016196 4355 ts_tablet_manager.cc:1397] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 22:46:56.016196 4354 ts_tablet_manager.cc:1397] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260807 22:46:56.025734 4356 raft_consensus.cc:348] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.026436 4356 raft_consensus.cc:374] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:56.026587 4356 raft_consensus.cc:729] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9599362e89cb41e69b48461b9360e657, State: Initialized, Role: FOLLOWER
I20260807 22:46:56.027552 4356 consensus_queue.cc:260] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.028723 4354 raft_consensus.cc:348] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.028976 4355 raft_consensus.cc:348] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.029274 4354 raft_consensus.cc:374] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:56.029390 4354 raft_consensus.cc:729] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157da2afcff64e15b4f79ed82acea0c0, State: Initialized, Role: FOLLOWER
I20260807 22:46:56.029428 4355 raft_consensus.cc:374] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:56.029517 4355 raft_consensus.cc:729] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfe291ece1f4e2ebd04064201e132aa, State: Initialized, Role: FOLLOWER
I20260807 22:46:56.030113 4355 consensus_queue.cc:260] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.030108 4354 consensus_queue.cc:260] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.032344 4354 ts_tablet_manager.cc:1428] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 22:46:56.032624 4355 ts_tablet_manager.cc:1428] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 22:46:56.033619 4192 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
I20260807 22:46:56.035008 4335 heartbeater.cc:503] Master 127.20.10.190:33961 was elected leader, sending a full tablet report...
I20260807 22:46:56.036293 4356 ts_tablet_manager.cc:1428] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Time spent starting tablet: real 0.020s user 0.010s sys 0.004s
W20260807 22:46:56.066211 4052 tablet.cc:2369] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:56.103724 4364 raft_consensus.cc:482] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:56.104076 4364 raft_consensus.cc:504] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.106307 4364 leader_election.cc:288] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625), 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921)
I20260807 22:46:56.113426 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.107043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:38283 (local address 127.20.10.129:40921)
0807 22:46:56.107281 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.107288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.107310 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:46:56.107399 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.107406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.107495 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.107598 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:56.108096 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.109222 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.111313 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.112507 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.112629 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.112687 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.113053 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.113153 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 22:46:56.113463 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.106545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.20.10.131:38283)
0807 22:46:56.106928 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.106962 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:56.107114 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:56.107651 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:56.107655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:56.107682 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:56.107900 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:56.107912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.109376 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.109382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:56.111134 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:56.111146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.112664 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.112670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:56.112874 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.112911 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:46:56.113013 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:46:56.113161 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260807 22:46:56.114612 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.106961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:56303 (local address 127.20.10.130:44625)
0807 22:46:56.107200 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.107209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.107898 (+ 689us) server_negotiation.cc:407] Connection header received
0807 22:46:56.108037 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.108042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.108120 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.108196 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:56.108966 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.110086 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.112394 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.113600 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.113773 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.113860 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.114176 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.114422 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:46:56.114686 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.106811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:44625 (local address 127.20.10.131:56303)
0807 22:46:56.107780 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.107797 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:56.107903 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:56.108372 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:56.108377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:56.108403 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:56.108779 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:56.108800 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.110256 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.110260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:56.112143 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:56.112160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.113859 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.113865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:56.114007 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.114031 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:46:56.114108 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:46:56.114275 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":536,"threads_started":1}
I20260807 22:46:56.114948 3999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb86d68fb04474ae1e0610e6be12f9" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9599362e89cb41e69b48461b9360e657" is_pre_election: true
I20260807 22:46:56.115569 3999 raft_consensus.cc:2454] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 0.
I20260807 22:46:56.116104 4143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb86d68fb04474ae1e0610e6be12f9" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157da2afcff64e15b4f79ed82acea0c0" is_pre_election: true
I20260807 22:46:56.116288 4215 leader_election.cc:302] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9599362e89cb41e69b48461b9360e657, adfe291ece1f4e2ebd04064201e132aa; no voters:
I20260807 22:46:56.116758 4364 raft_consensus.cc:2781] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:56.116693 4143 raft_consensus.cc:2454] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 0.
I20260807 22:46:56.116901 4364 raft_consensus.cc:482] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:56.117062 4364 raft_consensus.cc:3037] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:56.119654 4364 raft_consensus.cc:504] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.120311 4364 leader_election.cc:288] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 election: Requested vote from peers 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625), 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921)
I20260807 22:46:56.120918 3999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb86d68fb04474ae1e0610e6be12f9" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9599362e89cb41e69b48461b9360e657"
I20260807 22:46:56.120909 4143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8cb86d68fb04474ae1e0610e6be12f9" candidate_uuid: "adfe291ece1f4e2ebd04064201e132aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157da2afcff64e15b4f79ed82acea0c0"
I20260807 22:46:56.121142 3999 raft_consensus.cc:3037] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:56.121161 4143 raft_consensus.cc:3037] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:46:56.122864 4194 tablet.cc:2369] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:56.123803 3999 raft_consensus.cc:2454] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 1.
I20260807 22:46:56.124328 4215 leader_election.cc:302] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9599362e89cb41e69b48461b9360e657, adfe291ece1f4e2ebd04064201e132aa; no voters:
I20260807 22:46:56.124487 4143 raft_consensus.cc:2454] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfe291ece1f4e2ebd04064201e132aa in term 1.
I20260807 22:46:56.124722 4364 raft_consensus.cc:2781] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:56.125350 4364 raft_consensus.cc:686] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 1 LEADER]: Becoming Leader. State: Replica: adfe291ece1f4e2ebd04064201e132aa, State: Running, Role: LEADER
I20260807 22:46:56.125882 4364 consensus_queue.cc:237] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } }
I20260807 22:46:56.131664 3855 catalog_manager.cc:5495] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa reported cstate change: term changed from 0 to 1, leader changed from <none> to adfe291ece1f4e2ebd04064201e132aa (127.20.10.131). New cstate: current_term: 1 leader_uuid: "adfe291ece1f4e2ebd04064201e132aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfe291ece1f4e2ebd04064201e132aa" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 } health_report { overall_health: UNKNOWN } } }
W20260807 22:46:56.141564 4336 tablet.cc:2369] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:56.179311 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.173608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.20.10.131:33619)
0807 22:46:56.173831 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.173836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.173856 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:46:56.173958 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.173962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.174063 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.174224 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:56.174716 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.175528 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.177037 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.177476 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.177575 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.178099 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:56.178755 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:56.178871 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.179012 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.179091 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:46:56.189101 4143 raft_consensus.cc:1261] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Refusing update from remote peer adfe291ece1f4e2ebd04064201e132aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:56.189198 3999 raft_consensus.cc:1261] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Refusing update from remote peer adfe291ece1f4e2ebd04064201e132aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:56.190261 4366 consensus_queue.cc:1035] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "157da2afcff64e15b4f79ed82acea0c0" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:56.190646 4364 consensus_queue.cc:1035] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9599362e89cb41e69b48461b9360e657" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:56.227326 3855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:56.227658 3855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:56.232177 3855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Sending DeleteTablet for 3 replicas of tablet a8cb86d68fb04474ae1e0610e6be12f9
I20260807 22:46:56.233796 4266 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:56710
I20260807 22:46:56.233844 3978 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260807 22:46:56.234120 4122 tablet_service.cc:1514] Processing DeleteTablet for tablet a8cb86d68fb04474ae1e0610e6be12f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:56 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:50656
I20260807 22:46:56.235271 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3894
I20260807 22:46:56.247752 4377 tablet_replica.cc:330] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: stopping tablet replica
I20260807 22:46:56.248514 4377 raft_consensus.cc:2229] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:56.249256 4377 raft_consensus.cc:2258] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:56.252034 4377 ts_tablet_manager.cc:1905] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:56.257740 4379 tablet_replica.cc:330] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: stopping tablet replica
I20260807 22:46:56.258993 4379 raft_consensus.cc:2229] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:56.260455 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.236385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.20.10.129:40921)
0807 22:46:56.236569 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.236574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.236593 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:46:56.236776 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.236784 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.236901 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.237004 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:56.239451 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.241059 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.243729 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.244754 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.244909 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.249861 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:56.249888 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:56.249909 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:56.249997 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:56.255510 (+ 5513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.256801 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.256816 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.256822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.257070 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.258431 (+ 1361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.258437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.258440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.259037 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:56.259400 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.259713 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.259995 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:46:56.261229 4046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:56.262082 4378 tablet_replica.cc:330] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: stopping tablet replica
I20260807 22:46:56.262125 4379 raft_consensus.cc:2258] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:56.262813 4378 raft_consensus.cc:2229] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:56.263741 4378 raft_consensus.cc:2258] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:56.265151 4379 ts_tablet_manager.cc:1905] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:56.266847 4378 ts_tablet_manager.cc:1905] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:56.273290 4379 ts_tablet_manager.cc:1918] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:56.273574 4379 log.cc:1196] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/a8cb86d68fb04474ae1e0610e6be12f9
I20260807 22:46:56.274446 4379 ts_tablet_manager.cc:1939] T a8cb86d68fb04474ae1e0610e6be12f9 P 157da2afcff64e15b4f79ed82acea0c0: Deleting consensus metadata
I20260807 22:46:56.276690 4378 ts_tablet_manager.cc:1918] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:56.276868 3841 catalog_manager.cc:4841] TS 157da2afcff64e15b4f79ed82acea0c0 (127.20.10.130:44625): tablet a8cb86d68fb04474ae1e0610e6be12f9 (table test-table [id=6f7e1ab6fa024379b524bf2d6be777bc]) successfully deleted
I20260807 22:46:56.277034 4378 log.cc:1196] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/a8cb86d68fb04474ae1e0610e6be12f9
I20260807 22:46:56.277765 4378 ts_tablet_manager.cc:1939] T a8cb86d68fb04474ae1e0610e6be12f9 P adfe291ece1f4e2ebd04064201e132aa: Deleting consensus metadata
I20260807 22:46:56.279971 3841 catalog_manager.cc:4841] TS adfe291ece1f4e2ebd04064201e132aa (127.20.10.131:33619): tablet a8cb86d68fb04474ae1e0610e6be12f9 (table test-table [id=6f7e1ab6fa024379b524bf2d6be777bc]) successfully deleted
I20260807 22:46:56.372660 4377 ts_tablet_manager.cc:1918] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:56.372915 4377 log.cc:1196] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/a8cb86d68fb04474ae1e0610e6be12f9
I20260807 22:46:56.373464 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3894
I20260807 22:46:56.373543 4377 ts_tablet_manager.cc:1939] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Deleting consensus metadata
W20260807 22:46:56.403779 3838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.10.129:40921 (error 108)
I20260807 22:46:56.404147 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 4055
I20260807 22:46:56.406498 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.405954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58638)
0807 22:46:56.406160 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.406300 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 22:46:56.406962 3838 catalog_manager.cc:4584] TS 9599362e89cb41e69b48461b9360e657 (127.20.10.129:40921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8cb86d68fb04474ae1e0610e6be12f9: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
I20260807 22:46:56.420938 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.405295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.20.10.130:44625)
0807 22:46:56.405526 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.405534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.405557 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:46:56.405748 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.405754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.405859 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.405957 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:56.407971 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.408749 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.410138 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.410565 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.410737 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.413586 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:56.413605 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:56.413621 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:56.413670 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:56.417477 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.418481 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.418489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.418492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.418638 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.419143 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.419149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.419153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.419604 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:56.419943 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.420316 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.420567 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:46:56.421710 4185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:56.460949 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.460310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58652)
0807 22:46:56.460603 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.460754 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 22:46:56.462370 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.461858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58654)
0807 22:46:56.462074 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.462180 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 22:46:56.523293 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.522697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58660)
0807 22:46:56.522967 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.523088 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:46:56.524672 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.524221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58670)
0807 22:46:56.524405 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.524505 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 22:46:56.528481 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 4055
I20260807 22:46:56.552752 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 4197
I20260807 22:46:56.572883 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.553824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.20.10.131:33619)
0807 22:46:56.554061 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.554068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.554120 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:46:56.554197 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.554202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.554299 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.554435 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:56.556095 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.557389 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.559513 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.560233 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.560386 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.564104 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:56.564128 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:56.564151 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:56.564347 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:56.569043 (+ 4696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.570029 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.570040 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.570044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.570278 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.570873 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.570879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.570883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.571460 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:56.571844 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.572128 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.572408 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:46:56.573930 4328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:56.605873 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.605205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58672)
0807 22:46:56.605459 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.605584 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 22:46:56.607321 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.606810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:40921 (local address 127.0.0.1:58688)
0807 22:46:56.607050 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:56.607162 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 22:46:56.717077 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 4197
I20260807 22:46:56.745658 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 3821
I20260807 22:46:56.747109 3887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:46:56.868774 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 3821
I20260807 22:46:56.899149 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--webserver_interface=127.20.10.190
--webserver_port=33585
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:57.277591 4385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:57.277938 4385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:57.278034 4385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:57.285959 4385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:57.286116 4385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:57.286167 4385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:57.286221 4385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:57.286275 4385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:57.296959 4385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=33585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:57.298393 4385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:57.300186 4385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:57.315069 4390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:57.314962 4393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:57.314997 4391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:57.316025 4385 server_base.cc:1034] running on GCE node
I20260807 22:46:57.316972 4385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:57.318955 4385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:57.320276 4385 hybrid_clock.cc:648] HybridClock initialized: now 1786142817320226 us; error 54 us; skew 500 ppm
I20260807 22:46:57.320813 4385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:57 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:57.324689 4385 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:46:57.326505 4385 webserver.cc:460] Webserver started at http://127.20.10.190:33585/ using document root <none> and password file <none>
I20260807 22:46:57.327421 4385 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:57.327549 4385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:57.334291 4385 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:46:57.337785 4400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:57.339471 4385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 22:46:57.339684 4385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "aa49b0a062de43059eeb3d7769ffa141"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-csg5"
server_key: "652a4b21e954ef90783332d8d702b420"
server_key_iv: "defdf3afc97a355d5d8e030cfb8da2c7"
server_key_version: "encryptionkey@0"
I20260807 22:46:57.340852 4385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:57.360968 4385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:57.366266 4385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:57.366616 4385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:57.390200 4385 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33961
I20260807 22:46:57.390220 4457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33961 every 8 connection(s)
I20260807 22:46:57.392030 4385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:57.395481 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4385
I20260807 22:46:57.396581 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:40921
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35171
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:57.401944 4458 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:46:57.407737 4458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap starting.
I20260807 22:46:57.431526 4458 log.cc:824] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:57.450464 4458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:46:57.451129 4458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141: Bootstrap complete.
I20260807 22:46:57.461952 4458 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:57.463040 4458 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Initialized, Role: FOLLOWER
I20260807 22:46:57.463857 4458 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:57.464128 4458 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:57.464253 4458 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:57.464480 4458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:46:57.468886 4458 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:57.469489 4458 leader_election.cc:302] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa49b0a062de43059eeb3d7769ffa141; no voters:
I20260807 22:46:57.470242 4458 leader_election.cc:288] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:46:57.470407 4462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:46:57.472033 4462 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [term 3 LEADER]: Becoming Leader. State: Replica: aa49b0a062de43059eeb3d7769ffa141, State: Running, Role: LEADER
I20260807 22:46:57.472841 4458 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:57.472744 4462 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } }
I20260807 22:46:57.475943 4464 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa49b0a062de43059eeb3d7769ffa141. Latest consensus state: current_term: 3 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:57.476431 4464 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:57.476938 4463 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "aa49b0a062de43059eeb3d7769ffa141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49b0a062de43059eeb3d7769ffa141" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33961 } } }
I20260807 22:46:57.477269 4463 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49b0a062de43059eeb3d7769ffa141 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:57.488977 4469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:57.499399 4469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:57.501276 4469 catalog_manager.cc:1259] Loaded cluster ID: 0c380c4d449944c6a4d7d2652a858b19
I20260807 22:46:57.501479 4469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 22:46:57.504331 4469 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f25293d09ad google::LogMessage::Fail() at ??:0
@ 0x7f25293d388a google::LogMessage::SendToLog() at ??:0
@ 0x7f25293d04de google::LogMessage::Flush() at ??:0
@ 0x7f25293d1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f253ee03512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f253ee00976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f253ee88610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f253ee88439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f252c01c345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f252c01f91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f252c01f799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f252bffaeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f252f2d46db start_thread at ??:0
@ 0x7f252826a61f clone at ??:0
W20260807 22:46:57.862922 4460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:57.863232 4460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:57.863420 4460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:57.871155 4460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:57.871292 4460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:57.871492 4460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:46:57.881891 4460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:40921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=35171
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:57.883487 4460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:57.885200 4460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:57.900614 4505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:57.900516 4504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:57.900803 4508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:57.901562 4460 server_base.cc:1034] running on GCE node
I20260807 22:46:57.902382 4460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:57.903680 4460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:57.904961 4460 hybrid_clock.cc:648] HybridClock initialized: now 1786142817904883 us; error 82 us; skew 500 ppm
I20260807 22:46:57.905462 4460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:57 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:57.909439 4460 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:46:57.911376 4460 webserver.cc:460] Webserver started at http://127.20.10.129:35171/ using document root <none> and password file <none>
I20260807 22:46:57.912165 4460 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:57.912333 4460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:57.918474 4460 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:46:57.921764 4515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:57.923913 4460 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:46:57.924163 4460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "9599362e89cb41e69b48461b9360e657"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "cf60f9ed4864866cf34d79999b426152"
server_key_iv: "2c6d6ff94c9e927e2c0cc5c4b3d2e75c"
server_key_version: "encryptionkey@0"
I20260807 22:46:57.925494 4460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:57.957993 4460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:57.963852 4460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:57.964253 4460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:57.965651 4460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:57.968062 4522 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 22:46:57.974606 4522 ts_tablet_manager.cc:1852] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 22:46:57.975919 4522 ts_tablet_manager.cc:1905] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:57.985565 4522 ts_tablet_manager.cc:1918] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:57.985952 4522 ts_tablet_manager.cc:1939] T a8cb86d68fb04474ae1e0610e6be12f9 P 9599362e89cb41e69b48461b9360e657: Deleting consensus metadata
I20260807 22:46:57.986763 4460 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 22:46:57.986925 4460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.019s user 0.000s sys 0.001s
I20260807 22:46:57.987053 4460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:57.987124 4460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:58.030756 4460 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:40921
I20260807 22:46:58.030864 4637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:40921 every 8 connection(s)
I20260807 22:46:58.032823 4460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:58.038064 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4460
I20260807 22:46:58.038985 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:44625
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35185
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:58.039593 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.037049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:38605)
0807 22:46:58.038022 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.038347 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":454,"threads_started":1}
W20260807 22:46:58.040500 4638 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:33961 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:33961: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
I20260807 22:46:58.041774 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.041263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:39789)
0807 22:46:58.041475 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.041570 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 22:46:58.043262 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.042608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:51491)
0807 22:46:58.042849 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.042977 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 22:46:58.043601 4638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:46:58.440097 4641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:58.440451 4641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:58.440573 4641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:58.448580 4641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:58.448741 4641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:58.448865 4641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:46:58.459781 4641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:44625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=35185
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:58.461472 4641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:58.463129 4641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:58.479178 4646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:58.479274 4650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:58.479140 4648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:58.480650 4641 server_base.cc:1034] running on GCE node
I20260807 22:46:58.481482 4641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:58.482802 4641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:58.484108 4641 hybrid_clock.cc:648] HybridClock initialized: now 1786142818484035 us; error 63 us; skew 500 ppm
I20260807 22:46:58.484541 4641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:58 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:58.495071 4641 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:46:58.496999 4641 webserver.cc:460] Webserver started at http://127.20.10.130:35185/ using document root <none> and password file <none>
I20260807 22:46:58.497776 4641 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:58.497895 4641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:58.504251 4641 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:46:58.507910 4658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:58.509773 4641 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:58.509996 4641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "157da2afcff64e15b4f79ed82acea0c0"
format_stamp: "Formatted at 2026-08-07 22:46:51 on dist-test-slave-csg5"
server_key: "d2b9ea77335cc37a40e6cc6914f8ff4a"
server_key_iv: "0c8c845af12f77f2490071d562c94ac6"
server_key_version: "encryptionkey@0"
I20260807 22:46:58.511394 4641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:58.531490 4641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:58.536741 4641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:58.537073 4641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:58.538261 4641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:58.540045 4641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:58.540151 4641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:58.540289 4641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:58.540369 4641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:58.583458 4641 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:44625
I20260807 22:46:58.583516 4781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:44625 every 8 connection(s)
I20260807 22:46:58.585146 4641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:58.587232 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4641
I20260807 22:46:58.588307 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:33619
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:33961
--webserver_port=35057
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:58.591997 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.589197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:50107)
0807 22:46:58.590327 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.590626 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":360,"threads_started":1}
W20260807 22:46:58.592914 4783 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:33961 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:33961: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
I20260807 22:46:58.594089 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.593572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:59873)
0807 22:46:58.593797 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.593911 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 22:46:58.595341 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.594830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:34593)
0807 22:46:58.595024 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:58.595154 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 22:46:58.595657 4783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:46:58.972990 4788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:58.973372 4788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:58.973547 4788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:58.982277 4788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:58.982478 4788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:58.982769 4788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:46:58.994729 4788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:33619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=35057
--tserver_master_addrs=127.20.10.190:33961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:46:58.996260 4788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:58.997818 4788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:59.013676 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:59.013585 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:59.013634 4796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:59.014621 4788 server_base.cc:1034] running on GCE node
I20260807 22:46:59.015432 4788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:59.016777 4788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:59.018056 4788 hybrid_clock.cc:648] HybridClock initialized: now 1786142819017969 us; error 83 us; skew 500 ppm
I20260807 22:46:59.018543 4788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:46:59 dist-test-slave-csg5 krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142819, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:59.022193 4788 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:46:59.024115 4788 webserver.cc:460] Webserver started at http://127.20.10.131:35057/ using document root <none> and password file <none>
I20260807 22:46:59.025015 4788 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:59.025149 4788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:59.032245 4788 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 22:46:59.035878 4803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:59.038076 4788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:59.038324 4788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "adfe291ece1f4e2ebd04064201e132aa"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-csg5"
server_key: "c8322db236966b817e692f91d1a8caed"
server_key_iv: "8acdee17b2a663217ed12b4dad57ec2d"
server_key_version: "encryptionkey@0"
I20260807 22:46:59.039955 4788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:59.045368 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.044357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:45683)
0807 22:46:59.044980 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:59.045070 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":335,"threads_started":1}
I20260807 22:46:59.063043 4788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:59.068356 4788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:59.068631 4788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:59.069773 4788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:59.071359 4788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:59.071501 4788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:59.071600 4788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:59.071681 4788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:59.114638 4788 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33619
I20260807 22:46:59.114796 4926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33619 every 8 connection(s)
I20260807 22:46:59.116528 4788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:59.122293 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.120089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:59683)
0807 22:46:59.120993 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:59.121234 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":333,"threads_started":1}
W20260807 22:46:59.123276 4927 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:33961 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:33961: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
I20260807 22:46:59.124403 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.123936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:33083)
0807 22:46:59.124141 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:59.124230 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 22:46:59.125685 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.125152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:57149)
0807 22:46:59.125326 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:59.125455 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:46:59.126073 4927 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:46:59.126091 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 4788
I20260807 22:46:59.597311 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.596349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:60081)
0807 22:46:59.596905 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:59.597035 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":297,"threads_started":1}
I20260807 22:47:00.047611 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:00.046541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:56527)
0807 22:47:00.047276 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:00.047407 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":395,"threads_started":1}
I20260807 22:47:00.127939 4935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:00.126780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:45679)
0807 22:47:00.127476 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:00.127612 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":288,"threads_started":1}
I20260807 22:47:00.599772 4936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:00.598608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:40015)
0807 22:47:00.599392 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:00.599535 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":376,"threads_started":1}
I20260807 22:47:01.049674 4937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:01.048788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:51329)
0807 22:47:01.049375 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:01.049484 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":261,"threads_started":1}
I20260807 22:47:01.130411 4939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:01.129313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:42907)
0807 22:47:01.130015 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:01.130174 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":310,"threads_started":1}
I20260807 22:47:01.601766 4940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:01.600898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:43881)
0807 22:47:01.601465 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:01.601565 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":300,"threads_started":1}
I20260807 22:47:02.052057 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:02.050901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:48877)
0807 22:47:02.051537 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:02.051665 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":312,"threads_started":1}
I20260807 22:47:02.132519 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:02.131532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:56899)
0807 22:47:02.132076 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:02.132172 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":300,"threads_started":1}
I20260807 22:47:02.603811 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:02.602961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:36071)
0807 22:47:02.603486 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:02.603579 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":261,"threads_started":1}
I20260807 22:47:03.054706 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:03.053392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:54241)
0807 22:47:03.054230 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:03.054411 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":459,"threads_started":1}
I20260807 22:47:03.134832 4945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:03.133781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:37165)
0807 22:47:03.134379 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:03.134486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":306,"threads_started":1}
I20260807 22:47:03.606171 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:03.605071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:35793)
0807 22:47:03.605781 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:03.605913 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":368,"threads_started":1}
I20260807 22:47:04.057220 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:04.056002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:55189)
0807 22:47:04.056785 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:04.056963 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":389,"threads_started":1}
I20260807 22:47:04.136924 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:04.136073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:59621)
0807 22:47:04.136581 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:04.136674 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":259,"threads_started":1}
I20260807 22:47:04.608260 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:04.607449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:54267)
0807 22:47:04.607976 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:04.608080 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
I20260807 22:47:05.059346 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:05.058380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:47093)
0807 22:47:05.059024 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:05.059116 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":344,"threads_started":1}
I20260807 22:47:05.138916 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:05.138053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:43055)
0807 22:47:05.138555 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:05.138728 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260807 22:47:05.611037 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:05.609631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:33373)
0807 22:47:05.610408 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:05.610569 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":358,"threads_started":1}
I20260807 22:47:06.061481 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:06.060474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:47135)
0807 22:47:06.061141 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:06.061259 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":378,"threads_started":1}
I20260807 22:47:06.141403 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:06.140264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:45017)
0807 22:47:06.141059 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:06.141178 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":435,"threads_started":1}
I20260807 22:47:06.613260 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:06.612292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:53771)
0807 22:47:06.612868 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:06.612969 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":300,"threads_started":1}
I20260807 22:47:07.063673 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:07.062787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:33475)
0807 22:47:07.063358 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:07.063486 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":270,"threads_started":1}
I20260807 22:47:07.143683 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:07.142763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:52349)
0807 22:47:07.143389 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:07.143492 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":295,"threads_started":1}
I20260807 22:47:07.615774 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:07.614566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:55101)
0807 22:47:07.615295 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:07.615505 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":392,"threads_started":1}
I20260807 22:47:08.065923 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:08.064983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:42055)
0807 22:47:08.065630 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:08.065730 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":362,"threads_started":1}
I20260807 22:47:08.145787 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:08.144833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:48115)
0807 22:47:08.145350 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:08.145436 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":257,"threads_started":1}
I20260807 22:47:08.617807 4964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:08.616953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:45195)
0807 22:47:08.617508 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:08.617602 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":298,"threads_started":1}
I20260807 22:47:09.068006 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:09.067174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:34351)
0807 22:47:09.067714 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:09.067809 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":306,"threads_started":1}
I20260807 22:47:09.147876 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:09.147004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:44727)
0807 22:47:09.147537 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:09.147688 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":294,"threads_started":1}
I20260807 22:47:09.619956 4967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:09.619035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:55969)
0807 22:47:09.619667 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:09.619764 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":374,"threads_started":1}
I20260807 22:47:10.070212 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:10.069313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:43533)
0807 22:47:10.069913 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:10.070012 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":329,"threads_started":1}
I20260807 22:47:10.149914 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:10.148980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:59801)
0807 22:47:10.149588 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:10.149687 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":337,"threads_started":1}
I20260807 22:47:10.622051 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:10.621171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:53509)
0807 22:47:10.621770 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:10.621875 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":338,"threads_started":1}
I20260807 22:47:11.072571 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:11.071506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:41197)
0807 22:47:11.072210 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:11.072360 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":381,"threads_started":1}
I20260807 22:47:11.152727 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:11.151671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:45901)
0807 22:47:11.152404 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:11.152513 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":304,"threads_started":1}
I20260807 22:47:11.624548 4974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:11.623538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:46953)
0807 22:47:11.624189 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:11.624313 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":321,"threads_started":1}
I20260807 22:47:12.074797 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:12.073787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:34991)
0807 22:47:12.074389 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:12.074492 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":282,"threads_started":1}
I20260807 22:47:12.155016 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:12.154007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:33891)
0807 22:47:12.154607 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:12.154813 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":316,"threads_started":1}
I20260807 22:47:12.626753 4977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:12.625717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:35675)
0807 22:47:12.626345 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:12.626442 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":319,"threads_started":1}
I20260807 22:47:13.076972 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:13.076075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:50043)
0807 22:47:13.076649 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:13.076789 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":309,"threads_started":1}
I20260807 22:47:13.157019 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:13.156186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:40057)
0807 22:47:13.156667 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:13.156789 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260807 22:47:13.629156 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:13.628156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.130:50301)
0807 22:47:13.628751 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:13.628850 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":361,"threads_started":1}
I20260807 22:47:14.079147 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.078199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.129:44773)
0807 22:47:14.078847 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.078946 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":294,"threads_started":1}
I20260807 22:47:14.135169 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 4460
I20260807 22:47:14.159382 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.158481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33961 (local address 127.20.10.131:50093)
0807 22:47:14.159023 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.159115 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":308,"threads_started":1}
I20260807 22:47:14.164160 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.136479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33586 (local address 127.20.10.129:40921)
0807 22:47:14.137400 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:14.137427 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:14.137487 (+ 60us) server_negotiation.cc:407] Connection header received
0807 22:47:14.140071 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:14.140084 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:14.140617 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:14.141032 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:14.143053 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.144761 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.146414 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.147312 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.147554 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:14.155955 (+ 8401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:14.155966 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:14.155986 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:14.156046 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:14.160153 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.161072 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.161080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.161089 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.161293 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.161785 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.161791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.161799 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.162410 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:14.162792 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:14.163316 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 22:47:14.163829 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":489,"threads_started":1}
I20260807 22:47:14.166067 4631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:14.256971 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 4460
I20260807 22:47:14.282645 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 4641
I20260807 22:47:14.304922 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.284438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.20.10.130:44625)
0807 22:47:14.285026 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:14.285041 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:14.285080 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:47:14.286808 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:14.286819 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:14.287277 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:14.287655 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:14.289251 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.290718 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.292040 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.293090 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.293275 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:14.296950 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:14.296960 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:14.296979 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:14.297024 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:14.300494 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.301446 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.301456 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.301464 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.301693 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.302258 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.302263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.302265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.303091 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:14.303371 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:14.303943 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 22:47:14.304571 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":285,"threads_started":1}
I20260807 22:47:14.306627 4777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:14.405804 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 4641
I20260807 22:47:14.428809 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 4788
I20260807 22:47:14.457022 4987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.429986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.20.10.131:33619)
0807 22:47:14.430644 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:14.430713 (+ 69us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:14.430773 (+ 60us) server_negotiation.cc:407] Connection header received
0807 22:47:14.432690 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:14.432701 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:14.433168 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:14.433496 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:14.435479 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.436952 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.438595 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.439431 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.439601 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:14.447889 (+ 8288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:14.447900 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:14.447918 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:14.447971 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:14.452034 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.453100 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.453224 (+ 124us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.453234 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.453481 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.454170 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.454176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.454180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.455317 (+ 1137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:14.455663 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:14.456158 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 22:47:14.456684 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":310,"threads_started":1}
I20260807 22:47:14.458840 4922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:14.549940 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 4788
2026-08-07T22:47:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (26202 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:14 dist-test-slave-csg5 krb5kdc[4991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:14 dist-test-slave-csg5 krb5kdc[4991](info): set up 2 sockets
Aug 07 22:47:14 dist-test-slave-csg5 krb5kdc[4991](info): commencing operation
krb5kdc: starting...
W20260807 22:47:15.795785 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:47:16 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:47:16.152099 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:45979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:16.544457 5005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:16.544790 5005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:16.544986 5005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:16.552678 5005 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:47:16.552793 5005 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:47:16.552834 5005 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:47:16.552933 5005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:16.553030 5005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:16.553170 5005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:16.553234 5005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:16.553331 5005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:16.563964 5005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:45979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:16.565474 5005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:16.566972 5005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:16.581530 5014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:16.581588 5012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:16.581497 5011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:16.582118 5005 server_base.cc:1034] running on GCE node
I20260807 22:47:16.583425 5005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:16.585425 5005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:16.586812 5005 hybrid_clock.cc:648] HybridClock initialized: now 1786142836586702 us; error 52 us; skew 500 ppm
I20260807 22:47:16.587297 5005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:16 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:16.591522 5005 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:47:16.593309 5005 webserver.cc:460] Webserver started at http://127.20.10.190:34397/ using document root <none> and password file <none>
I20260807 22:47:16.594089 5005 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:16.594195 5005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:16.594484 5005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:16.598016 5005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "c7f726106e0444b3b69ea148c916b219"
format_stamp: "Formatted at 2026-08-07 22:47:16 on dist-test-slave-csg5"
server_key: "ccffa5cd5455a6aa4b04ea46d4a746ea"
server_key_iv: "edb07741aa428ef70c52432c74b54019"
server_key_version: "encryptionkey@0"
I20260807 22:47:16.598937 5005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "c7f726106e0444b3b69ea148c916b219"
format_stamp: "Formatted at 2026-08-07 22:47:16 on dist-test-slave-csg5"
server_key: "ccffa5cd5455a6aa4b04ea46d4a746ea"
server_key_iv: "edb07741aa428ef70c52432c74b54019"
server_key_version: "encryptionkey@0"
I20260807 22:47:16.604946 5005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:47:16.609593 5022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:16.611490 5005 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:47:16.611701 5005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "c7f726106e0444b3b69ea148c916b219"
format_stamp: "Formatted at 2026-08-07 22:47:16 on dist-test-slave-csg5"
server_key: "ccffa5cd5455a6aa4b04ea46d4a746ea"
server_key_iv: "edb07741aa428ef70c52432c74b54019"
server_key_version: "encryptionkey@0"
I20260807 22:47:16.611958 5005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:16.636222 5005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:16.641360 5005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:16.641680 5005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:16.663456 5005 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:45979
I20260807 22:47:16.663479 5081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:45979 every 8 connection(s)
I20260807 22:47:16.664961 5005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:16.667999 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5005
I20260807 22:47:16.668287 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:47:16.669036 20522 external_mini_cluster.cc:1428] Setting key e6d58fe77e7f8c80612ec06cfe8d6cc0
I20260807 22:47:16.669236 5082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:16.686909 5082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap starting.
Aug 07 22:47:16 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:16.691823 5082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:16.693537 5082 log.cc:824] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:16.698137 5082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: No bootstrap required, opened a new log
I20260807 22:47:16.700445 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.671190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.20.10.190:45979)
0807 22:47:16.672056 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.672068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.672129 (+ 61us) server_negotiation.cc:407] Connection header received
0807 22:47:16.674579 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.674609 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.675238 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.675658 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:16.677931 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.679679 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.681709 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.682745 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.682937 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.688972 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:16.688988 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:16.689012 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:16.689094 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:16.694026 (+ 4932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:16.695504 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:16.695516 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:16.695525 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:16.695823 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:16.696382 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:16.696387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:16.696390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:16.697301 (+ 911us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:16.697654 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.698204 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.698814 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":408,"threads_started":1}
I20260807 22:47:16.708848 5082 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:16.709306 5082 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:16.709391 5082 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Initialized, Role: FOLLOWER
I20260807 22:47:16.710026 5082 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:16.710269 5082 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:16.710387 5082 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:16.710585 5082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:16.713346 5082 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:16.713934 5082 leader_election.cc:302] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f726106e0444b3b69ea148c916b219; no voters:
I20260807 22:47:16.714727 5082 leader_election.cc:288] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:16.714915 5087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:16.715776 5087 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 LEADER]: Becoming Leader. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Running, Role: LEADER
I20260807 22:47:16.716470 5087 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:16.717334 5082 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:16.719795 5088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:47:16.720363 5088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:16.721417 5095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:16.721616 5089 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f726106e0444b3b69ea148c916b219. Latest consensus state: current_term: 1 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:47:16.722069 5089 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:16.727909 5095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:16.740582 5095 catalog_manager.cc:1347] Generated new cluster ID: e027ea0912a34df6b134f54a7e445c6e
I20260807 22:47:16.740762 5095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:16.760185 5095 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:16.761935 5095 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:16.769161 5095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Generated new TSK 0
I20260807 22:47:16.770359 5095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:47:16.868443 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:17.246002 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:17.246345 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:17.246510 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:17.254390 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:17.254527 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:17.254751 5110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:47:17.265559 5110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:17.267042 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:17.269011 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:17.284592 5116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.284505 5120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.284484 5118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:17.286597 5110 server_base.cc:1034] running on GCE node
I20260807 22:47:17.287421 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:17.288722 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:17.290019 5110 hybrid_clock.cc:648] HybridClock initialized: now 1786142837289954 us; error 43 us; skew 500 ppm
I20260807 22:47:17.290447 5110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:17 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:17.294144 5110 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:47:17.295881 5110 webserver.cc:460] Webserver started at http://127.20.10.129:45209/ using document root <none> and password file <none>
I20260807 22:47:17.296588 5110 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:17.296698 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:17.296977 5110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:17.300448 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "c7ec7df9883a40ca99f1534a89befd0d"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "f38984ff0faf8e58ff99b4cbff8741cb"
server_key_iv: "938f6edb2778eefae366a8de5c61ba6a"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.301291 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "c7ec7df9883a40ca99f1534a89befd0d"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "f38984ff0faf8e58ff99b4cbff8741cb"
server_key_iv: "938f6edb2778eefae366a8de5c61ba6a"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.307829 5110 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:47:17.312541 5127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.314450 5110 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:47:17.314787 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "c7ec7df9883a40ca99f1534a89befd0d"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "f38984ff0faf8e58ff99b4cbff8741cb"
server_key_iv: "938f6edb2778eefae366a8de5c61ba6a"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.315069 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:17.352449 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:17.357656 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:17.358012 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:17.359292 5110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:17.360862 5110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:17.360940 5110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.361089 5110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:17.361150 5110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.400501 5110 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:33065
I20260807 22:47:17.400559 5253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:33065 every 8 connection(s)
I20260807 22:47:17.401945 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:47:17.407133 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5110
I20260807 22:47:17.407403 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:47:17.408345 20522 external_mini_cluster.cc:1428] Setting key d9a3aed52585a472d5b39ee1d5ad6be1
Aug 07 22:47:17 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:17.431483 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:17.405123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:58097 (local address 127.20.10.190:45979)
0807 22:47:17.405616 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:17.405621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:17.406913 (+ 1292us) server_negotiation.cc:407] Connection header received
0807 22:47:17.409857 (+ 2944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:17.409862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:17.409956 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:17.410084 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:17.412615 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:17.413597 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:17.416783 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:17.417458 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:17.417670 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:17.423570 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:17.423577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:17.423581 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:17.423633 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:17.427129 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:17.428377 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:17.428385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:17.428389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:17.428575 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:17.429182 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:17.429187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:17.429189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:17.429532 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:17.429726 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:17.430733 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0807 22:47:17.431048 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
I20260807 22:47:17.432739 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:17.405757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:58097)
0807 22:47:17.406679 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:17.406742 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:17.409476 (+ 2734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:17.410420 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:17.410430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:17.411336 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:17.412334 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:17.412385 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:17.413831 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:17.413840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:17.416517 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:17.416540 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:17.417777 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:17.417787 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:17.418162 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:17.419102 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:17.419167 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:17.423163 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:17.427371 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:17.427389 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:17.427407 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:17.428186 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:17.428732 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:17.428738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:17.428742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:17.428994 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:17.429698 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:17.429708 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:17.430175 (+ 467us) client_negotiation.cc:769] Sending connection context
0807 22:47:17.430554 (+ 379us) client_negotiation.cc:240] Negotiation successful
0807 22:47:17.431248 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":343,"threads_started":1}
I20260807 22:47:17.434957 5254 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:17.435341 5254 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:17.436256 5254 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:17.439141 5041 ts_manager.cc:194] Registered new tserver with Master: c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129:33065)
I20260807 22:47:17.441653 5041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:58097
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:47:17.491637 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:17.874312 5260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:17.874748 5260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:17.874953 5260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:17.883612 5260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:17.883796 5260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:17.883986 5260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:47:17.894796 5260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:17.896323 5260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:17.897843 5260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:17.912760 5265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.912796 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.912846 5266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:17.914363 5260 server_base.cc:1034] running on GCE node
I20260807 22:47:17.915155 5260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:17.916461 5260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:17.917732 5260 hybrid_clock.cc:648] HybridClock initialized: now 1786142837917696 us; error 47 us; skew 500 ppm
I20260807 22:47:17.918216 5260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:17 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:17.922163 5260 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:47:17.924098 5260 webserver.cc:460] Webserver started at http://127.20.10.130:36551/ using document root <none> and password file <none>
I20260807 22:47:17.924889 5260 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:17.925050 5260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:17.925390 5260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:17.928937 5260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "0b72b6f875ce43efb74b3ad442eb5570"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "5fe4e161cc580a9354018c004cb9460e"
server_key_iv: "a5b25da6e73dba1d74743565c2d14170"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.929829 5260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "0b72b6f875ce43efb74b3ad442eb5570"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "5fe4e161cc580a9354018c004cb9460e"
server_key_iv: "a5b25da6e73dba1d74743565c2d14170"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.936208 5260 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:47:17.940183 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.941992 5260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:47:17.942278 5260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "0b72b6f875ce43efb74b3ad442eb5570"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "5fe4e161cc580a9354018c004cb9460e"
server_key_iv: "a5b25da6e73dba1d74743565c2d14170"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.942479 5260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:17.968808 5260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:17.973649 5260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:17.973922 5260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:17.975020 5260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:17.976516 5260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:17.976600 5260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.976689 5260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:17.976732 5260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.013182 5260 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35813
I20260807 22:47:18.013226 5406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35813 every 8 connection(s)
I20260807 22:47:18.014542 5260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:47:18.018370 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5260
I20260807 22:47:18.018618 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:47:18.019228 20522 external_mini_cluster.cc:1428] Setting key 75cecb4be67220b97e2ba62a66936c24
Aug 07 22:47:18 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:18.041041 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.017766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:56239 (local address 127.20.10.190:45979)
0807 22:47:18.018421 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.018428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.020210 (+ 1782us) server_negotiation.cc:407] Connection header received
0807 22:47:18.022310 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.022316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.022412 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.022557 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.024479 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.025324 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.027720 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.028201 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.028363 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.032866 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.032872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.032875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.032915 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.036511 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.037860 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.037865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.037869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.038040 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.038846 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.038850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.038853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.039218 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.039371 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.040515 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.040746 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":339,"threads_started":1}
I20260807 22:47:18.042744 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.018471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:56239)
0807 22:47:18.019669 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.019734 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.021989 (+ 2255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.022876 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.022886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.023575 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.024265 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.024284 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.025528 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.025532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.027520 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.027536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.028479 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.028487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.028774 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.029611 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.029638 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.032506 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.036753 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.036764 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.036784 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.037640 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.038182 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.038190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.038196 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.038622 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.039398 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.039410 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.039986 (+ 576us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.040369 (+ 383us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.041019 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":490,"threads_started":1}
I20260807 22:47:18.044723 5407 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:18.045094 5407 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:18.046041 5407 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:18.047928 5041 ts_manager.cc:194] Registered new tserver with Master: 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813)
I20260807 22:47:18.049337 5041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:56239
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:47:18.103588 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:18.445997 5254 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
W20260807 22:47:18.480598 5417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:18.480975 5417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:18.481166 5417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:18.489152 5417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:18.489362 5417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:18.489557 5417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:47:18.500156 5417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:18.501684 5417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:18.503165 5417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:18.517769 5422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.517722 5423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.518019 5425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:18.518689 5417 server_base.cc:1034] running on GCE node
I20260807 22:47:18.519459 5417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:18.520542 5417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:18.521821 5417 hybrid_clock.cc:648] HybridClock initialized: now 1786142838521728 us; error 56 us; skew 500 ppm
I20260807 22:47:18.522171 5417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:18 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:18.525842 5417 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:47:18.527693 5417 webserver.cc:460] Webserver started at http://127.20.10.131:35761/ using document root <none> and password file <none>
I20260807 22:47:18.528496 5417 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:18.528652 5417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:18.528992 5417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:18.532330 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "257188db28464f6cb64ce736f6fadcf6"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-csg5"
server_key: "99d16a042a0c9b9318dc2a29e6a0c84c"
server_key_iv: "50b31ea141b63a1453cade7e7adf4a05"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.533253 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "257188db28464f6cb64ce736f6fadcf6"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-csg5"
server_key: "99d16a042a0c9b9318dc2a29e6a0c84c"
server_key_iv: "50b31ea141b63a1453cade7e7adf4a05"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.539707 5417 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 22:47:18.544023 5432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.546309 5417 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 22:47:18.546490 5417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "257188db28464f6cb64ce736f6fadcf6"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-csg5"
server_key: "99d16a042a0c9b9318dc2a29e6a0c84c"
server_key_iv: "50b31ea141b63a1453cade7e7adf4a05"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.546871 5417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:18.568199 5417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:18.573429 5417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:18.573726 5417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:18.574937 5417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:18.576412 5417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:18.576491 5417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.576586 5417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:18.576630 5417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.615329 5417 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33537
I20260807 22:47:18.615449 5559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33537 every 8 connection(s)
I20260807 22:47:18.616991 5417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:47:18.620545 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5417
I20260807 22:47:18.620788 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:47:18.621448 20522 external_mini_cluster.cc:1428] Setting key b3fb402e0026b1b932f60003cc8ae266
Aug 07 22:47:18 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:18.640977 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.620111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:34591 (local address 127.20.10.190:45979)
0807 22:47:18.620622 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.620629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.621985 (+ 1356us) server_negotiation.cc:407] Connection header received
0807 22:47:18.624158 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.624164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.624278 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.624394 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.626078 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.627075 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.628883 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.629338 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.629443 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.633731 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.633737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.633740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.633780 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.637197 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.638313 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.638317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.638319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.638488 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.639016 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.639020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.639022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.639327 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.639449 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.640442 (+ 993us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.640648 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":244,"threads_started":1}
I20260807 22:47:18.641994 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.620822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:34591)
0807 22:47:18.621790 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.621840 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.623778 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.624697 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.624705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.625315 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.625902 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.625920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.627233 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.627237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.628712 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.628728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.629581 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.629587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.629858 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.630631 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.630776 (+ 145us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.633400 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.637384 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.637397 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.637422 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.638154 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.638602 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.638605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.638607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.638903 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.639486 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.639494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.639903 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.640260 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.640710 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":387,"threads_started":1}
I20260807 22:47:18.643718 5560 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:18.644008 5560 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:18.644795 5560 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:18.646560 5041 ts_manager.cc:194] Registered new tserver with Master: 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537)
I20260807 22:47:18.647512 5041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:34591
I20260807 22:47:18.650934 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:47:18.672065 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.656143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.20.10.190:45979)
0807 22:47:18.656468 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.656476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.656831 (+ 355us) server_negotiation.cc:407] Connection header received
0807 22:47:18.657058 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.657064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.657177 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.657312 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.658986 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.659750 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.661167 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.661595 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.661756 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.665137 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.665142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.665146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.665187 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.668486 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.669550 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.669555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.669557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.669706 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.670350 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.670355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.670357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.670896 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.671142 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.671578 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.671820 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:47:18.677557 5041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:47:18.685988 5041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:47:18.727182 5573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.702457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:43372)
0807 22:47:18.715850 (+ 13393us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.715890 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.716088 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.717789 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.717793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.717828 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:18.718320 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.718333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.720772 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.720777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.723548 (+ 2771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.723570 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.726010 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.726017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.726237 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.726283 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.726400 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.726567 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13224,"mutex_wait_us":34,"thread_start_us":14764,"threads_started":1}
I20260807 22:47:18.727607 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.702902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35813 (local address 127.0.0.1:34686)
0807 22:47:18.717661 (+ 14759us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.717693 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.717993 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.718980 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.718985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.719018 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:18.719608 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.719622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.722001 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.722005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.724030 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.724047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.726317 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.726324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.726649 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.726746 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.726833 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.726996 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14575,"mutex_wait_us":74,"thread_start_us":8159,"threads_started":1}
I20260807 22:47:18.727588 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.702435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.20.10.131:33537)
0807 22:47:18.715485 (+ 13050us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.715501 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.715560 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:47:18.715734 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.715750 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.716281 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.716607 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:18.718259 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.720161 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.723968 (+ 3807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.725654 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.725857 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.726136 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.726956 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.727142 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12751,"thread_start_us":12502,"threads_started":1}
I20260807 22:47:18.727669 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.704907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.20.10.130:35813)
0807 22:47:18.710210 (+ 5303us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.710225 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.717844 (+ 7619us) server_negotiation.cc:407] Connection header received
0807 22:47:18.718246 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.718253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.718442 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.718576 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:18.719908 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.721751 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.724246 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.725750 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.725898 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.726092 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.726946 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.727248 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4984,"thread_start_us":4777,"threads_started":1}
I20260807 22:47:18.728483 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.702025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33537 (local address 127.0.0.1:41710)
0807 22:47:18.711637 (+ 9612us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.711704 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.711981 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.716956 (+ 4975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.716966 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.717021 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:18.717734 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.717756 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.720439 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.720445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.723752 (+ 3307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.723763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.726330 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.726335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.726644 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.726742 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.726835 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.728015 (+ 1180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9318,"mutex_wait_us":1175,"thread_start_us":13486,"threads_started":1}
I20260807 22:47:18.729264 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.704081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.20.10.129:33065)
0807 22:47:18.716987 (+ 12906us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.716999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.717042 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:47:18.717179 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.717193 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.717529 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.718551 (+ 1022us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:18.718873 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.720578 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.723802 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.725706 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.728266 (+ 2560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.728600 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.728697 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.728893 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12664,"thread_start_us":12421,"threads_started":1}
I20260807 22:47:18.732560 5483 tablet_service.cc:1467] Processing CreateTablet for tablet 11ecdb0e8abc4790be86769856d467e8 (DEFAULT_TABLE table=test-table [id=d973d805715242aba28c958f0f545068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:18.732560 5333 tablet_service.cc:1467] Processing CreateTablet for tablet 11ecdb0e8abc4790be86769856d467e8 (DEFAULT_TABLE table=test-table [id=d973d805715242aba28c958f0f545068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:18.733114 5183 tablet_service.cc:1467] Processing CreateTablet for tablet 11ecdb0e8abc4790be86769856d467e8 (DEFAULT_TABLE table=test-table [id=d973d805715242aba28c958f0f545068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:18.734249 5333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ecdb0e8abc4790be86769856d467e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.734246 5483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ecdb0e8abc4790be86769856d467e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.735074 5183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ecdb0e8abc4790be86769856d467e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.750494 5580 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Bootstrap starting.
I20260807 22:47:18.754871 5580 tablet_bootstrap.cc:654] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.755745 5581 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Bootstrap starting.
I20260807 22:47:18.756901 5580 log.cc:824] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.759089 5583 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Bootstrap starting.
I20260807 22:47:18.762209 5581 tablet_bootstrap.cc:654] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.763540 5583 tablet_bootstrap.cc:654] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.763979 5581 log.cc:824] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.764590 5580 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: No bootstrap required, opened a new log
I20260807 22:47:18.764973 5580 ts_tablet_manager.cc:1397] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260807 22:47:18.765164 5583 log.cc:824] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.768038 5581 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: No bootstrap required, opened a new log
I20260807 22:47:18.768325 5583 tablet_bootstrap.cc:492] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: No bootstrap required, opened a new log
I20260807 22:47:18.768380 5581 ts_tablet_manager.cc:1397] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260807 22:47:18.768635 5583 ts_tablet_manager.cc:1397] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 22:47:18.777683 5583 raft_consensus.cc:348] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.778335 5583 raft_consensus.cc:374] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:18.778452 5583 raft_consensus.cc:729] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b72b6f875ce43efb74b3ad442eb5570, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.778157 5580 raft_consensus.cc:348] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.778710 5580 raft_consensus.cc:374] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:18.778831 5580 raft_consensus.cc:729] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ec7df9883a40ca99f1534a89befd0d, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.779515 5583 consensus_queue.cc:260] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.779541 5580 consensus_queue.cc:260] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.781450 5581 raft_consensus.cc:348] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.781898 5580 ts_tablet_manager.cc:1428] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Time spent starting tablet: real 0.017s user 0.010s sys 0.009s
I20260807 22:47:18.782037 5581 raft_consensus.cc:374] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:18.782151 5581 raft_consensus.cc:729] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257188db28464f6cb64ce736f6fadcf6, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.782964 5581 consensus_queue.cc:260] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.783352 5407 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
I20260807 22:47:18.784176 5583 ts_tablet_manager.cc:1428] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260807 22:47:18.786074 5560 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
I20260807 22:47:18.788175 5581 ts_tablet_manager.cc:1428] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Time spent starting tablet: real 0.020s user 0.019s sys 0.001s
I20260807 22:47:18.834326 5588 raft_consensus.cc:482] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:47:18.834743 5588 raft_consensus.cc:504] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.836819 5588 leader_election.cc:288] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537), 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813)
I20260807 22:47:18.844116 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.837439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:42975 (local address 127.20.10.130:35813)
0807 22:47:18.837703 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.837709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.838342 (+ 633us) server_negotiation.cc:407] Connection header received
0807 22:47:18.838607 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.838613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.838773 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.838884 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:18.839536 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.840289 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.842138 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.843085 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.843286 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.843370 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.843669 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.843838 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 22:47:18.844255 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.837210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35813 (local address 127.20.10.129:42975)
0807 22:47:18.838193 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.838214 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.838433 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.838911 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.838916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.838946 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:18.839379 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.839390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.840434 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.840440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.841958 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.841974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.843306 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.843312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.843493 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.843519 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.843641 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.843801 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":3,"spinlock_wait_cycles":55040,"thread_start_us":652,"threads_started":1}
I20260807 22:47:18.845402 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.837348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:35725 (local address 127.20.10.131:33537)
0807 22:47:18.837589 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.837595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.838999 (+ 1404us) server_negotiation.cc:407] Connection header received
0807 22:47:18.839269 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.839275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.839375 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.839492 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:18.840114 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.840936 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.843221 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.844184 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.844400 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.844486 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.844985 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.845137 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 22:47:18.845517 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.837475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33537 (local address 127.20.10.129:35725)
0807 22:47:18.838844 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.838863 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.839090 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.839511 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.839516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.839544 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:18.839935 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.839945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.841105 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.841123 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.843033 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.843047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.844657 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.844664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.844818 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.844841 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.844954 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.845106 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":30,"spinlock_wait_cycles":1280,"thread_start_us":871,"threads_started":1}
I20260807 22:47:18.845306 5354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ecdb0e8abc4790be86769856d467e8" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b72b6f875ce43efb74b3ad442eb5570" is_pre_election: true
I20260807 22:47:18.845983 5354 raft_consensus.cc:2454] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 0.
I20260807 22:47:18.846858 5130 leader_election.cc:302] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b72b6f875ce43efb74b3ad442eb5570, c7ec7df9883a40ca99f1534a89befd0d; no voters:
I20260807 22:47:18.846760 5511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ecdb0e8abc4790be86769856d467e8" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257188db28464f6cb64ce736f6fadcf6" is_pre_election: true
I20260807 22:47:18.847347 5588 raft_consensus.cc:2781] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:47:18.847371 5511 raft_consensus.cc:2454] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 0.
I20260807 22:47:18.847584 5588 raft_consensus.cc:482] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:47:18.847726 5588 raft_consensus.cc:3037] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.850793 5588 raft_consensus.cc:504] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.851662 5588 leader_election.cc:288] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 election: Requested vote from peers 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537), 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813)
I20260807 22:47:18.852361 5354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ecdb0e8abc4790be86769856d467e8" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b72b6f875ce43efb74b3ad442eb5570"
I20260807 22:47:18.852375 5511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ecdb0e8abc4790be86769856d467e8" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257188db28464f6cb64ce736f6fadcf6"
I20260807 22:47:18.852669 5354 raft_consensus.cc:3037] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.852677 5511 raft_consensus.cc:3037] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.855369 5511 raft_consensus.cc:2454] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 1.
I20260807 22:47:18.855402 5354 raft_consensus.cc:2454] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 1.
I20260807 22:47:18.855973 5128 leader_election.cc:302] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257188db28464f6cb64ce736f6fadcf6, c7ec7df9883a40ca99f1534a89befd0d; no voters:
I20260807 22:47:18.856364 5588 raft_consensus.cc:2781] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:18.856989 5588 raft_consensus.cc:686] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 1 LEADER]: Becoming Leader. State: Replica: c7ec7df9883a40ca99f1534a89befd0d, State: Running, Role: LEADER
I20260807 22:47:18.857539 5588 consensus_queue.cc:237] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } }
I20260807 22:47:18.862787 5038 catalog_manager.cc:5495] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d reported cstate change: term changed from 0 to 1, leader changed from <none> to c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129). New cstate: current_term: 1 leader_uuid: "c7ec7df9883a40ca99f1534a89befd0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } health_report { overall_health: UNKNOWN } } }
W20260807 22:47:18.869004 5561 tablet.cc:2369] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:18.893036 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.886763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.20.10.129:33065)
0807 22:47:18.886988 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.886997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.887039 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:47:18.887420 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.887424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.887510 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.887703 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:18.888159 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.888961 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.890554 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.891057 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.891205 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.891819 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:18.892389 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:18.892566 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.892649 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.892741 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:47:18.903167 5354 raft_consensus.cc:1261] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Refusing update from remote peer c7ec7df9883a40ca99f1534a89befd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:18.903167 5511 raft_consensus.cc:1261] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Refusing update from remote peer c7ec7df9883a40ca99f1534a89befd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:18.904471 5588 consensus_queue.cc:1035] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:18.904912 5592 consensus_queue.cc:1035] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:47:18.905144 5255 tablet.cc:2369] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:18.919114 5595 mvcc.cc:205] Tried to move back new op lower bound from 7316041068135677952 to 7316041067965321216. Current Snapshot: MvccSnapshot[applied={T|T < 7316041068135677952}]
I20260807 22:47:18.920123 5596 mvcc.cc:205] Tried to move back new op lower bound from 7316041068135677952 to 7316041067965321216. Current Snapshot: MvccSnapshot[applied={T|T < 7316041068135677952}]
I20260807 22:47:18.935302 5041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:47:18.935626 5041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:47:18.939821 5041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Sending DeleteTablet for 3 replicas of tablet 11ecdb0e8abc4790be86769856d467e8
I20260807 22:47:18.941254 5183 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ecdb0e8abc4790be86769856d467e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:18 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:43372
I20260807 22:47:18.941243 5483 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ecdb0e8abc4790be86769856d467e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:18 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:41710
I20260807 22:47:18.941274 5333 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ecdb0e8abc4790be86769856d467e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:18 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:34686
I20260807 22:47:18.942848 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5110
I20260807 22:47:18.973201 5604 tablet_replica.cc:330] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: stopping tablet replica
I20260807 22:47:18.973934 5604 raft_consensus.cc:2229] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:47:18.980070 5604 raft_consensus.cc:2258] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:18.981352 5606 tablet_replica.cc:330] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: stopping tablet replica
I20260807 22:47:18.981611 5605 tablet_replica.cc:330] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: stopping tablet replica
I20260807 22:47:18.981907 5606 raft_consensus.cc:2229] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:47:18.982053 5605 raft_consensus.cc:2229] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:47:18.982538 5606 raft_consensus.cc:2258] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:18.982878 5605 raft_consensus.cc:2258] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:18.983573 5604 ts_tablet_manager.cc:1905] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:47:18.985038 5606 ts_tablet_manager.cc:1905] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:47:18 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:47:18.991775 5604 ts_tablet_manager.cc:1918] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:18.991770 5605 ts_tablet_manager.cc:1905] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:47:18.991993 5604 log.cc:1196] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/11ecdb0e8abc4790be86769856d467e8
I20260807 22:47:18.992614 5604 ts_tablet_manager.cc:1939] T 11ecdb0e8abc4790be86769856d467e8 P 0b72b6f875ce43efb74b3ad442eb5570: Deleting consensus metadata
I20260807 22:47:18.994841 5606 ts_tablet_manager.cc:1918] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:18.994807 5026 catalog_manager.cc:4841] TS 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813): tablet 11ecdb0e8abc4790be86769856d467e8 (table test-table [id=d973d805715242aba28c958f0f545068]) successfully deleted
I20260807 22:47:18.995059 5606 log.cc:1196] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/11ecdb0e8abc4790be86769856d467e8
I20260807 22:47:18.995690 5606 ts_tablet_manager.cc:1939] T 11ecdb0e8abc4790be86769856d467e8 P 257188db28464f6cb64ce736f6fadcf6: Deleting consensus metadata
I20260807 22:47:18.997938 5023 catalog_manager.cc:4841] TS 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537): tablet 11ecdb0e8abc4790be86769856d467e8 (table test-table [id=d973d805715242aba28c958f0f545068]) successfully deleted
I20260807 22:47:19.002884 5605 ts_tablet_manager.cc:1918] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:19.003135 5605 log.cc:1196] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/11ecdb0e8abc4790be86769856d467e8
I20260807 22:47:19.003814 5605 ts_tablet_manager.cc:1939] T 11ecdb0e8abc4790be86769856d467e8 P c7ec7df9883a40ca99f1534a89befd0d: Deleting consensus metadata
I20260807 22:47:19.006281 5024 catalog_manager.cc:4841] TS c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129:33065): tablet 11ecdb0e8abc4790be86769856d467e8 (table test-table [id=d973d805715242aba28c958f0f545068]) successfully deleted
I20260807 22:47:19.006950 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.944446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.20.10.129:33065)
0807 22:47:18.944666 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.944676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.967901 (+ 23225us) server_negotiation.cc:407] Connection header received
0807 22:47:18.968620 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.968630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.968765 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.968868 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.974969 (+ 6101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.976759 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.983336 (+ 6577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.984197 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.984333 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.996812 (+ 12479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.996832 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.996858 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.996925 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:19.002352 (+ 5427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.003757 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.003773 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.003778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.004032 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.004590 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.004597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.004602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.005189 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:19.005588 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.005979 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.006309 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:47:19.007777 5249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:19.132220 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5110
I20260807 22:47:19.156805 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5260
Aug 07 22:47:19 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:47:19.180593 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.157704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.20.10.130:35813)
0807 22:47:19.157931 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.157937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.157956 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:47:19.158211 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.158215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.158297 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.158393 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:19.159940 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.160824 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.162378 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.163016 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.163244 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.174235 (+ 10991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:19.174258 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:19.174274 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:19.174325 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:19.177463 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.178321 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.178329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.178331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.178468 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.178905 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.178908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.178910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.179488 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:19.179734 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.180129 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.180331 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 22:47:19.181418 5398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:19.276887 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5260
I20260807 22:47:19.301301 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5417
Aug 07 22:47:19 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:47:19.319034 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.302220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.20.10.131:33537)
0807 22:47:19.302493 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.302500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.302521 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:47:19.302729 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.302734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.302802 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.302915 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:19.304389 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.305151 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.306628 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.307262 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.307413 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.312392 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:19.312411 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:19.312428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:19.312480 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:19.315714 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.316575 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.316583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.316586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.316731 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:19.317200 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:19.317206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:19.317209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:19.317631 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:19.317987 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.318371 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.318611 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 22:47:19.319926 5553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:19.417871 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5417
I20260807 22:47:19.440912 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5005
I20260807 22:47:19.442253 5076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:19.565770 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5005
I20260807 22:47:39.597157 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--webserver_interface=127.20.10.190
--webserver_port=34397
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:45979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:39.978581 5613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:39.979028 5613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:39.979228 5613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:39.987025 5613 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:47:39.987211 5613 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:47:39.987288 5613 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:47:39.987365 5613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:39.987448 5613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:39.987550 5613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:39.987641 5613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:39.987725 5613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:39.998390 5613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:45979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=34397
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:39.999917 5613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:40.001439 5613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:40.032078 5623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:40.032120 5621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:40.032109 5620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:40.033097 5613 server_base.cc:1034] running on GCE node
I20260807 22:47:40.034116 5613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:40.036047 5613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:40.037408 5613 hybrid_clock.cc:648] HybridClock initialized: now 1786142860037302 us; error 75 us; skew 500 ppm
I20260807 22:47:40.037923 5613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:40 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142860, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:40.047889 5613 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:47:40.050271 5613 webserver.cc:460] Webserver started at http://127.20.10.190:34397/ using document root <none> and password file <none>
I20260807 22:47:40.051354 5613 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:40.051541 5613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:40.059320 5613 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:47:40.063120 5632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:40.065364 5613 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:47:40.065640 5613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "c7f726106e0444b3b69ea148c916b219"
format_stamp: "Formatted at 2026-08-07 22:47:16 on dist-test-slave-csg5"
server_key: "ccffa5cd5455a6aa4b04ea46d4a746ea"
server_key_iv: "edb07741aa428ef70c52432c74b54019"
server_key_version: "encryptionkey@0"
I20260807 22:47:40.066942 5613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:40.098129 5613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:40.103384 5613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:40.103757 5613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:40.127005 5613 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:45979
I20260807 22:47:40.127043 5689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:45979 every 8 connection(s)
I20260807 22:47:40.128816 5613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:40.136229 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5613
I20260807 22:47:40.137493 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:33065
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=45209
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:40.138443 5690 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:47:40.145025 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap starting.
I20260807 22:47:40.169622 5690 log.cc:824] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:40.184130 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:47:40.184825 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap complete.
I20260807 22:47:40.195004 5690 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:40.195976 5690 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Initialized, Role: FOLLOWER
I20260807 22:47:40.196671 5690 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:40.196972 5690 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:40.197154 5690 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:40.197351 5690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:47:40.202080 5690 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:40.202940 5690 leader_election.cc:302] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f726106e0444b3b69ea148c916b219; no voters:
I20260807 22:47:40.203729 5690 leader_election.cc:288] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:47:40.203912 5694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:47:40.206670 5694 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 LEADER]: Becoming Leader. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Running, Role: LEADER
I20260807 22:47:40.207316 5694 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:47:40.207593 5690 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:40.210726 5697 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f726106e0444b3b69ea148c916b219. Latest consensus state: current_term: 2 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:47:40.210745 5696 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:47:40.211373 5696 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:40.211401 5697 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:40.223305 5700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:40.239171 5700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:40.241590 5700 catalog_manager.cc:1259] Loaded cluster ID: e027ea0912a34df6b134f54a7e445c6e
I20260807 22:47:40.241775 5700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:40.248261 5700 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:40.250694 5700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Loaded TSK: 0
I20260807 22:47:40.294948 5700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Generated new TSK 1
I20260807 22:47:40.295353 5700 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:47:40.582779 5692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:40.583127 5692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:40.583323 5692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:40.592823 5692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:40.592976 5692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:40.593184 5692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:47:40.605051 5692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:33065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=45209
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:40.606699 5692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:40.608608 5692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:40.626534 5724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:40.626545 5720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:40.627457 5719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:40.628748 5692 server_base.cc:1034] running on GCE node
I20260807 22:47:40.629513 5692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:40.630647 5692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:40.631969 5692 hybrid_clock.cc:648] HybridClock initialized: now 1786142860631898 us; error 63 us; skew 500 ppm
I20260807 22:47:40.632480 5692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:40 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142860, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:40.636305 5692 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:47:40.638561 5692 webserver.cc:460] Webserver started at http://127.20.10.129:45209/ using document root <none> and password file <none>
I20260807 22:47:40.639386 5692 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:40.639546 5692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:40.646229 5692 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 22:47:40.649501 5731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:40.651237 5692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 22:47:40.651511 5692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "c7ec7df9883a40ca99f1534a89befd0d"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "f38984ff0faf8e58ff99b4cbff8741cb"
server_key_iv: "938f6edb2778eefae366a8de5c61ba6a"
server_key_version: "encryptionkey@0"
I20260807 22:47:40.652854 5692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:40.675603 5692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:40.680967 5692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:40.681303 5692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:40.682549 5692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:40.684135 5692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:40.684269 5692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:40.684415 5692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:40.684521 5692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:40.731305 5692 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:33065
I20260807 22:47:40.731410 5857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:33065 every 8 connection(s)
I20260807 22:47:40.732990 5692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:47:40.737479 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5692
I20260807 22:47:40.738193 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:35813
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=36551
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:47:40 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142860, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:40.763047 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:40.736538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:41779 (local address 127.20.10.190:45979)
0807 22:47:40.737387 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:40.737406 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:40.738596 (+ 1190us) server_negotiation.cc:407] Connection header received
0807 22:47:40.740695 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:40.740717 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:40.741211 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:40.741612 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:40.743297 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:40.745531 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:40.747601 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:40.748887 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:40.749141 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:40.753625 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:40.753634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:40.753651 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:40.753725 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:40.757668 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:40.758705 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:40.758719 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:40.758729 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:40.758974 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:40.759495 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:40.759500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:40.759504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:40.760050 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:40.760277 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:40.761137 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 22:47:40.761542 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":315,"threads_started":1}
I20260807 22:47:40.763036 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:40.737107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:41779)
0807 22:47:40.738071 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:40.738129 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:40.740078 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:40.741765 (+ 1687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:40.741773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:40.742369 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:40.743069 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:40.743088 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:40.745802 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:40.745806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:40.747410 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:40.747427 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:40.749209 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:40.749217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:40.749466 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:40.750274 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:40.750299 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:40.753255 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:40.757892 (+ 4637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:40.757902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:40.757920 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:40.758480 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:40.759120 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:40.759125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:40.759129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:40.759359 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:40.760217 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:40.760225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:40.760609 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 22:47:40.760898 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 22:47:40.761542 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":378,"threads_started":1}
I20260807 22:47:40.766837 5858 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:40.767421 5858 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:40.768674 5858 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:40.771747 5653 ts_manager.cc:194] Registered new tserver with Master: c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129:33065)
I20260807 22:47:40.773633 5653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:41779
W20260807 22:47:41.141536 5863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:41.141876 5863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:41.142063 5863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:41.151059 5863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:41.151227 5863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:41.151417 5863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:47:41.163715 5863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:35813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=36551
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:41.165505 5863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:41.167107 5863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:41.183166 5869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:41.183215 5870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:41.183145 5873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:41.184792 5863 server_base.cc:1034] running on GCE node
I20260807 22:47:41.185899 5863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:41.187203 5863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:41.188516 5863 hybrid_clock.cc:648] HybridClock initialized: now 1786142861188439 us; error 69 us; skew 500 ppm
I20260807 22:47:41.188956 5863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:41 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142861, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:41.193262 5863 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:47:41.195550 5863 webserver.cc:460] Webserver started at http://127.20.10.130:36551/ using document root <none> and password file <none>
I20260807 22:47:41.196481 5863 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:41.196699 5863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:41.204852 5863 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 22:47:41.208917 5881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.212078 5863 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 22:47:41.212497 5863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "0b72b6f875ce43efb74b3ad442eb5570"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "5fe4e161cc580a9354018c004cb9460e"
server_key_iv: "a5b25da6e73dba1d74743565c2d14170"
server_key_version: "encryptionkey@0"
I20260807 22:47:41.214059 5863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:41.252983 5863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:41.258013 5863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:41.258306 5863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:41.259685 5863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:41.261358 5863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:41.261469 5863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.261554 5863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:41.261610 5863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.303687 5863 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35813
I20260807 22:47:41.303867 6004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35813 every 8 connection(s)
I20260807 22:47:41.305464 5863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:47:41.311021 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 5863
I20260807 22:47:41.311918 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:33537
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=35761
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:47:41 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142861, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:41.333382 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.308607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:44723 (local address 127.20.10.190:45979)
0807 22:47:41.309228 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.309234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.310311 (+ 1077us) server_negotiation.cc:407] Connection header received
0807 22:47:41.313288 (+ 2977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.313294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.313382 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.313502 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:41.315998 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.316808 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.319717 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.320349 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.320472 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.325403 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:41.325409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:41.325412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:41.325451 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:41.329595 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.330599 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.330604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.330606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.330852 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.331362 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.331372 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.331374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.331718 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:41.331897 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.332700 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.332988 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":268,"threads_started":1}
I20260807 22:47:41.334542 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.309165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:44723)
0807 22:47:41.310019 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:41.310068 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:41.312874 (+ 2806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:41.313841 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:41.313856 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:41.314800 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:41.315741 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.315773 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.317015 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.317021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.319365 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.319389 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.320643 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.320654 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.321025 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.321943 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:41.321980 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:41.325061 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:41.329828 (+ 4767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:41.329837 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:41.329853 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:41.330423 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:41.330983 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:41.330989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:41.330992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:41.331221 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:41.331867 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:41.331876 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:41.332295 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 22:47:41.332584 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 22:47:41.333222 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":325,"threads_started":1}
I20260807 22:47:41.336716 6005 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:41.337121 6005 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:41.338003 6005 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:41.339953 5653 ts_manager.cc:194] Registered new tserver with Master: 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813)
I20260807 22:47:41.340860 5653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:44723
W20260807 22:47:41.689042 6010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:41.689589 6010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:41.689827 6010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:41.697739 6010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:41.697958 6010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:41.698159 6010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:47:41.710090 6010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:33537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=35761
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:47:41.711964 6010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:41.713619 6010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:41.729048 6016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:41.729066 6018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:41.729271 6015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:41.730046 6010 server_base.cc:1034] running on GCE node
I20260807 22:47:41.730911 6010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:41.732178 6010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:41.733417 6010 hybrid_clock.cc:648] HybridClock initialized: now 1786142861733379 us; error 40 us; skew 500 ppm
I20260807 22:47:41.733850 6010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:47:41 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142861, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:41.737619 6010 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:47:41.739820 6010 webserver.cc:460] Webserver started at http://127.20.10.131:35761/ using document root <none> and password file <none>
I20260807 22:47:41.740622 6010 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:41.740798 6010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:41.746877 6010 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:47:41.750063 6027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.751616 6010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:47:41.751838 6010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "257188db28464f6cb64ce736f6fadcf6"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-csg5"
server_key: "99d16a042a0c9b9318dc2a29e6a0c84c"
server_key_iv: "50b31ea141b63a1453cade7e7adf4a05"
server_key_version: "encryptionkey@0"
I20260807 22:47:41.752934 6010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:41.771271 6010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:41.776113 6010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:41.776432 6010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:41.777583 6010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:47:41.778189 5858 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
I20260807 22:47:41.779366 6010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:47:41.779488 6010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.779594 6010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:47:41.779685 6010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:41.818221 6010 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33537
I20260807 22:47:41.818262 6147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33537 every 8 connection(s)
I20260807 22:47:41.819806 6010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:47:41.821311 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6010
Aug 07 22:47:41 dist-test-slave-csg5 krb5kdc[4991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142861, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:47:41.840852 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.822751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.20.10.190:45979)
0807 22:47:41.823004 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.823013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.823543 (+ 530us) server_negotiation.cc:407] Connection header received
0807 22:47:41.823774 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.823779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.823860 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.823969 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:41.826410 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.827407 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.829560 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.830394 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.830539 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.833850 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:41.833856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:41.833859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:41.833900 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:41.837277 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.838295 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.838300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.838303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.838517 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.839038 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.839043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.839046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.839546 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:41.839871 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.840249 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.840507 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:47:41.845153 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.823130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:36385 (local address 127.20.10.190:45979)
0807 22:47:41.823671 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.823677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.824929 (+ 1252us) server_negotiation.cc:407] Connection header received
0807 22:47:41.827419 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.827426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.827521 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.827637 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:41.829909 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.831155 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.832947 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.833396 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.833536 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.837785 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:41.837792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:41.837796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:41.837845 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:41.840997 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.842049 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.842052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.842055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.842193 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.842837 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.842843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.842848 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.843261 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:41.843484 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.844533 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.844808 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":236,"threads_started":1}
I20260807 22:47:41.846300 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.823805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:36385)
0807 22:47:41.824715 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:41.824776 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:41.827143 (+ 2367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:41.827960 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:41.827972 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:41.828833 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:41.829681 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.829706 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.831331 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.831336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.832779 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.832794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.833650 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.833661 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.833946 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.834778 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:41.834805 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:41.837521 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:41.841223 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:41.841238 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:41.841259 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:41.841918 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:41.842332 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:41.842337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:41.842341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:41.842630 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:41.843445 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:41.843459 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:41.843981 (+ 522us) client_negotiation.cc:769] Sending connection context
0807 22:47:41.844432 (+ 451us) client_negotiation.cc:240] Negotiation successful
0807 22:47:41.844911 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":354,"threads_started":1}
I20260807 22:47:41.848269 6148 heartbeater.cc:347] Connected to a master server at 127.20.10.190:45979
I20260807 22:47:41.848657 6148 heartbeater.cc:464] Registering TS with master...
I20260807 22:47:41.849646 6148 heartbeater.cc:511] Master 127.20.10.190:45979 requested a full tablet report, sending...
I20260807 22:47:41.851385 5653 ts_manager.cc:194] Registered new tserver with Master: 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537)
I20260807 22:47:41.852394 5653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:36385
I20260807 22:47:41.857823 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:47:41.865861 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.859100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.20.10.190:45979)
0807 22:47:41.859354 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.859364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.859956 (+ 592us) server_negotiation.cc:407] Connection header received
0807 22:47:41.860224 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.860227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.860404 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.860538 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:41.861070 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.861951 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.863630 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.864092 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.864197 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.864908 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:41.865236 (+ 328us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 22:47:41.865555 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53848: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 22:47:41.866051 6151 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53848: token expired
W20260807 22:47:41.867060 5566 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.10.190:45979)
I20260807 22:47:41.882820 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.867743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.20.10.190:45979)
0807 22:47:41.867949 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.867953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.867971 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:47:41.868122 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.868126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.868194 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.868403 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:41.869998 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.871095 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.872704 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.873174 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.873301 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.876406 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:41.876412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:41.876415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:41.876457 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:41.879548 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.880442 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.880447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.880450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.880653 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:41.881232 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:41.881236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:41.881239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:41.881593 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:41.881837 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.882241 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.882456 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:47:41.887058 5653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:47:41.895397 5653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:47:41.933714 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.910459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.20.10.130:35813)
0807 22:47:41.923357 (+ 12898us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.923371 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.923414 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:47:41.923549 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.923562 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.923913 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.924184 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:41.926266 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.927834 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.930760 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.932206 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.932398 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.932655 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.933099 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.933312 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12716,"thread_start_us":12602,"threads_started":1}
I20260807 22:47:41.934095 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.909720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58692)
0807 22:47:41.920708 (+ 10988us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:41.920788 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:41.921032 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:41.925711 (+ 4679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:41.925716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:41.925746 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:41.926242 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.926259 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.928102 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.928106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.930413 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.930429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.932533 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.932540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.932814 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.932878 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 22:47:41.932990 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:47:41.933320 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10587,"mutex_wait_us":157,"thread_start_us":12541,"threads_started":1}
I20260807 22:47:41.934137 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.909762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35813 (local address 127.0.0.1:46842)
0807 22:47:41.922575 (+ 12813us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:41.922597 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:41.922771 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:41.924722 (+ 1951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:41.924740 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:41.924783 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:41.925980 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.925994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.927996 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.928005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.930500 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.930517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.932488 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.932495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.932849 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.932883 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:47:41.933037 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 22:47:41.933303 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12681,"mutex_wait_us":169,"spinlock_wait_cycles":7040,"thread_start_us":16256,"threads_started":1}
I20260807 22:47:41.934827 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.910167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33537 (local address 127.0.0.1:43406)
0807 22:47:41.925447 (+ 15280us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:41.925467 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:41.925637 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:41.926467 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:41.926471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:41.926498 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:41.926935 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.926949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.928987 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.928995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.931482 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:41.931499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.933922 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.933929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:41.934117 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.934164 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:47:41.934254 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:47:41.934409 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15151,"spinlock_wait_cycles":10496,"thread_start_us":11150,"threads_started":1}
I20260807 22:47:41.935391 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.910061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.20.10.129:33065)
0807 22:47:41.924944 (+ 14883us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.924956 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.924983 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:47:41.925107 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.925118 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.925477 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.926335 (+ 858us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:41.926627 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.927903 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.930642 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.932244 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.934554 (+ 2310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.934909 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.934999 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.935148 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14704,"thread_start_us":14540,"threads_started":1}
I20260807 22:47:41.935401 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:41.911007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.20.10.131:33537)
0807 22:47:41.925768 (+ 14761us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:41.925779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:41.925815 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:47:41.925922 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:41.925931 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:41.926162 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:41.926373 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:41.927133 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.928826 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.931683 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:41.933342 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:41.933487 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:41.933747 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:41.934853 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0807 22:47:41.935034 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14282,"thread_start_us":14717,"threads_started":1}
I20260807 22:47:41.938397 5781 tablet_service.cc:1467] Processing CreateTablet for tablet 05a969cbfd34461692160224529e160c (DEFAULT_TABLE table=test-table [id=1dc518a9954c469f81a490feb70e746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:41.938409 5933 tablet_service.cc:1467] Processing CreateTablet for tablet 05a969cbfd34461692160224529e160c (DEFAULT_TABLE table=test-table [id=1dc518a9954c469f81a490feb70e746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:41.940037 5781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a969cbfd34461692160224529e160c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:41.940045 5933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a969cbfd34461692160224529e160c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:41.942984 6078 tablet_service.cc:1467] Processing CreateTablet for tablet 05a969cbfd34461692160224529e160c (DEFAULT_TABLE table=test-table [id=1dc518a9954c469f81a490feb70e746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:47:41.944772 6078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a969cbfd34461692160224529e160c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:41.955030 6165 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Bootstrap starting.
I20260807 22:47:41.960427 6165 tablet_bootstrap.cc:654] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:41.961712 6165 log.cc:824] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:41.962785 6166 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Bootstrap starting.
I20260807 22:47:41.964757 6168 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Bootstrap starting.
I20260807 22:47:41.967690 6166 tablet_bootstrap.cc:654] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:41.968745 6168 tablet_bootstrap.cc:654] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:41.969570 6165 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: No bootstrap required, opened a new log
I20260807 22:47:41.969573 6166 log.cc:824] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:41.969861 6168 log.cc:824] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:41.969911 6165 ts_tablet_manager.cc:1397] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260807 22:47:41.973305 6166 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: No bootstrap required, opened a new log
I20260807 22:47:41.973640 6166 ts_tablet_manager.cc:1397] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 22:47:41.973788 6168 tablet_bootstrap.cc:492] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: No bootstrap required, opened a new log
I20260807 22:47:41.974072 6168 ts_tablet_manager.cc:1397] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 22:47:41.981801 6165 raft_consensus.cc:348] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.982358 6165 raft_consensus.cc:374] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:41.982481 6165 raft_consensus.cc:729] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ec7df9883a40ca99f1534a89befd0d, State: Initialized, Role: FOLLOWER
I20260807 22:47:41.983245 6165 consensus_queue.cc:260] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.984059 6168 raft_consensus.cc:348] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.984622 6168 raft_consensus.cc:374] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:41.984733 6168 raft_consensus.cc:729] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257188db28464f6cb64ce736f6fadcf6, State: Initialized, Role: FOLLOWER
I20260807 22:47:41.985410 6165 ts_tablet_manager.cc:1428] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
W20260807 22:47:41.985697 5859 tablet.cc:2369] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:41.985764 6168 consensus_queue.cc:260] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.985116 6166 raft_consensus.cc:348] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.986563 6166 raft_consensus.cc:374] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:41.986881 6166 raft_consensus.cc:729] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b72b6f875ce43efb74b3ad442eb5570, State: Initialized, Role: FOLLOWER
I20260807 22:47:41.987483 6148 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
I20260807 22:47:41.987862 6166 consensus_queue.cc:260] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:41.988394 6168 ts_tablet_manager.cc:1428] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260807 22:47:41.989678 6005 heartbeater.cc:503] Master 127.20.10.190:45979 was elected leader, sending a full tablet report...
I20260807 22:47:41.990530 6166 ts_tablet_manager.cc:1428] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260807 22:47:42.057981 6006 tablet.cc:2369] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:47:42.071684 6149 tablet.cc:2369] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:42.240497 6172 raft_consensus.cc:482] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:47:42.241040 6172 raft_consensus.cc:504] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:42.243230 6172 leader_election.cc:288] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813), 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537)
I20260807 22:47:42.251585 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.243883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:60893 (local address 127.20.10.131:33537)
0807 22:47:42.244156 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.244165 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.245296 (+ 1131us) server_negotiation.cc:407] Connection header received
0807 22:47:42.245560 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.245566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.245641 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.245761 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:42.246414 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.247258 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.249457 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.250322 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.250557 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.250646 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.251117 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.251285 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:47:42.251606 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.243811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:42983 (local address 127.20.10.130:35813)
0807 22:47:42.244074 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.244084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.245313 (+ 1229us) server_negotiation.cc:407] Connection header received
0807 22:47:42.245556 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.245562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.245634 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.245740 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:47:42.246449 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.247257 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.249518 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.250354 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.250555 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.250689 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.251117 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.251285 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:47:42.251817 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.243667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:33537 (local address 127.20.10.129:60893)
0807 22:47:42.245130 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.245150 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:42.245406 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:42.245777 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:42.245782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:42.245808 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:42.246249 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:42.246262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.247409 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.247416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:42.249246 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:42.249263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.250600 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.250607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:42.250909 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.250962 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:47:42.251084 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 22:47:42.251285 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"mutex_wait_us":112,"spinlock_wait_cycles":56448,"thread_start_us":714,"threads_started":1}
I20260807 22:47:42.251870 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.243576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:35813 (local address 127.20.10.129:42983)
0807 22:47:42.245175 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.245191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:42.245396 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:42.245815 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:42.245820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:42.245845 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:47:42.246274 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:42.246283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.247400 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.247407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:42.249255 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:42.249276 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.250639 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.250646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:42.250913 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.250964 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 22:47:42.251084 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 22:47:42.251285 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1389,"mutex_wait_us":198,"spinlock_wait_cycles":4864,"thread_start_us":974,"threads_started":1}
I20260807 22:47:42.253139 6098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a969cbfd34461692160224529e160c" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257188db28464f6cb64ce736f6fadcf6" is_pre_election: true
I20260807 22:47:42.253144 5956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a969cbfd34461692160224529e160c" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b72b6f875ce43efb74b3ad442eb5570" is_pre_election: true
I20260807 22:47:42.253783 6098 raft_consensus.cc:2454] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 0.
I20260807 22:47:42.253788 5956 raft_consensus.cc:2454] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 0.
I20260807 22:47:42.254544 5734 leader_election.cc:302] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b72b6f875ce43efb74b3ad442eb5570, 257188db28464f6cb64ce736f6fadcf6, c7ec7df9883a40ca99f1534a89befd0d; no voters:
I20260807 22:47:42.255117 6172 raft_consensus.cc:2781] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:47:42.255358 6172 raft_consensus.cc:482] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:47:42.255483 6172 raft_consensus.cc:3037] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:42.258448 6172 raft_consensus.cc:504] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:42.259266 6172 leader_election.cc:288] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 election: Requested vote from peers 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813), 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537)
I20260807 22:47:42.259954 5956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a969cbfd34461692160224529e160c" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b72b6f875ce43efb74b3ad442eb5570"
I20260807 22:47:42.260210 5956 raft_consensus.cc:3037] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:42.262878 5956 raft_consensus.cc:2454] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 1.
I20260807 22:47:42.262921 6098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a969cbfd34461692160224529e160c" candidate_uuid: "c7ec7df9883a40ca99f1534a89befd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257188db28464f6cb64ce736f6fadcf6"
I20260807 22:47:42.263201 6098 raft_consensus.cc:3037] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:42.263818 5734 leader_election.cc:302] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b72b6f875ce43efb74b3ad442eb5570, c7ec7df9883a40ca99f1534a89befd0d; no voters:
I20260807 22:47:42.264240 6172 raft_consensus.cc:2781] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:42.264806 6172 raft_consensus.cc:686] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 1 LEADER]: Becoming Leader. State: Replica: c7ec7df9883a40ca99f1534a89befd0d, State: Running, Role: LEADER
I20260807 22:47:42.265630 6172 consensus_queue.cc:237] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } }
I20260807 22:47:42.267151 6098 raft_consensus.cc:2454] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ec7df9883a40ca99f1534a89befd0d in term 1.
I20260807 22:47:42.271804 5652 catalog_manager.cc:5495] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d reported cstate change: term changed from 0 to 1, leader changed from <none> to c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129). New cstate: current_term: 1 leader_uuid: "c7ec7df9883a40ca99f1534a89befd0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ec7df9883a40ca99f1534a89befd0d" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 } health_report { overall_health: UNKNOWN } } }
I20260807 22:47:42.301860 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.295301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.20.10.129:33065)
0807 22:47:42.295551 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.295556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.295727 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:47:42.295880 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.295884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.295987 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.296128 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:42.296614 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.297656 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.299345 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.299804 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.299917 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.300546 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:42.301246 (+ 700us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:42.301370 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.301557 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.301642 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:47:42.311682 6098 raft_consensus.cc:1261] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Refusing update from remote peer c7ec7df9883a40ca99f1534a89befd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:42.311687 5956 raft_consensus.cc:1261] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Refusing update from remote peer c7ec7df9883a40ca99f1534a89befd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:42.313002 6172 consensus_queue.cc:1035] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b72b6f875ce43efb74b3ad442eb5570" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 35813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:42.313369 6178 consensus_queue.cc:1035] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "257188db28464f6cb64ce736f6fadcf6" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 33537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:42.327201 6180 mvcc.cc:205] Tried to move back new op lower bound from 7316041164016406528 to 7316041163846529024. Current Snapshot: MvccSnapshot[applied={T|T < 7316041164016406528}]
I20260807 22:47:42.334931 6182 mvcc.cc:205] Tried to move back new op lower bound from 7316041164016406528 to 7316041163846529024. Current Snapshot: MvccSnapshot[applied={T|T < 7316041164016406528}]
I20260807 22:47:42.355399 5652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:47:42.355798 5652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:47:42.361220 5652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Sending DeleteTablet for 3 replicas of tablet 05a969cbfd34461692160224529e160c
I20260807 22:47:42.362854 6078 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a969cbfd34461692160224529e160c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:43406
I20260807 22:47:42.362865 5781 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a969cbfd34461692160224529e160c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:58692
I20260807 22:47:42.362862 5933 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a969cbfd34461692160224529e160c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:47:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:46842
I20260807 22:47:42.363973 6192 tablet_replica.cc:330] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: stopping tablet replica
I20260807 22:47:42.364635 6192 raft_consensus.cc:2229] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:47:42.365026 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5692
I20260807 22:47:42.365545 6192 raft_consensus.cc:2258] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:42.366422 6190 tablet_replica.cc:330] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: stopping tablet replica
I20260807 22:47:42.366604 6191 tablet_replica.cc:330] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: stopping tablet replica
I20260807 22:47:42.366998 6190 raft_consensus.cc:2229] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:47:42.367316 6191 raft_consensus.cc:2229] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:47:42.367946 6191 raft_consensus.cc:2258] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:42.367971 6190 raft_consensus.cc:2258] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:47:42.370731 6191 ts_tablet_manager.cc:1905] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:47:42.370729 6192 ts_tablet_manager.cc:1905] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:47:42.378908 6191 ts_tablet_manager.cc:1918] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:42.379245 6191 log.cc:1196] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/05a969cbfd34461692160224529e160c
I20260807 22:47:42.379614 6192 ts_tablet_manager.cc:1918] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:42.379830 6192 log.cc:1196] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/05a969cbfd34461692160224529e160c
I20260807 22:47:42.380206 6191 ts_tablet_manager.cc:1939] T 05a969cbfd34461692160224529e160c P 0b72b6f875ce43efb74b3ad442eb5570: Deleting consensus metadata
I20260807 22:47:42.380471 6192 ts_tablet_manager.cc:1939] T 05a969cbfd34461692160224529e160c P 257188db28464f6cb64ce736f6fadcf6: Deleting consensus metadata
I20260807 22:47:42.382834 5633 catalog_manager.cc:4841] TS 257188db28464f6cb64ce736f6fadcf6 (127.20.10.131:33537): tablet 05a969cbfd34461692160224529e160c (table test-table [id=1dc518a9954c469f81a490feb70e746e]) successfully deleted
I20260807 22:47:42.383315 5635 catalog_manager.cc:4841] TS 0b72b6f875ce43efb74b3ad442eb5570 (127.20.10.130:35813): tablet 05a969cbfd34461692160224529e160c (table test-table [id=1dc518a9954c469f81a490feb70e746e]) successfully deleted
I20260807 22:47:42.385042 6190 ts_tablet_manager.cc:1905] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:47:42.391667 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.366879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.20.10.129:33065)
0807 22:47:42.367126 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.367134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.369076 (+ 1942us) server_negotiation.cc:407] Connection header received
0807 22:47:42.369409 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.369416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.369511 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.369591 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:42.371832 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.373078 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.375369 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.376100 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.376205 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.380790 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:42.380823 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:42.380847 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:42.380903 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:42.387718 (+ 6815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.388838 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.388849 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.388853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.389077 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.389608 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.389617 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.389621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.390178 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:42.390550 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.390986 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.391269 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:47:42.392627 5852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:42.392745 6190 ts_tablet_manager.cc:1918] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:47:42.392966 6190 log.cc:1196] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/05a969cbfd34461692160224529e160c
I20260807 22:47:42.500432 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5692
I20260807 22:47:42.525687 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5863
W20260807 22:47:42.525817 5634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.10.129:33065 (error 108)
I20260807 22:47:42.528270 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.527459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58706)
0807 22:47:42.527881 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.527989 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":229,"threads_started":1}
W20260807 22:47:42.528514 5634 catalog_manager.cc:4584] TS c7ec7df9883a40ca99f1534a89befd0d (127.20.10.129:33065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05a969cbfd34461692160224529e160c: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
I20260807 22:47:42.542973 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.526660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.20.10.130:35813)
0807 22:47:42.526880 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.526887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.527027 (+ 140us) server_negotiation.cc:407] Connection header received
0807 22:47:42.527291 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.527296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.527372 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.527458 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:42.529314 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.530104 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.531963 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.532399 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.532505 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.536148 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:42.536164 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:42.536180 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:42.536219 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:42.539581 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.540564 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.540572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.540575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.540719 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.541256 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.541260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.541262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.541663 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:42.541894 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.542421 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.542609 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:47:42.543742 6000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:42.582615 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.581991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58720)
0807 22:47:42.582282 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.582423 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 22:47:42.584017 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.583538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58726)
0807 22:47:42.583723 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.583820 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 22:47:42.642431 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5863
I20260807 22:47:42.644871 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.644187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58736)
0807 22:47:42.644550 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.644667 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260807 22:47:42.646118 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.645630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58746)
0807 22:47:42.645831 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.645932 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:47:42.665629 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6010
I20260807 22:47:42.682214 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.666524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.20.10.131:33537)
0807 22:47:42.666975 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:42.666984 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:42.667013 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:47:42.667088 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:42.667095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:42.667182 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:42.667308 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:42.668947 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.669790 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.671553 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:42.672024 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:42.672269 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:42.675372 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:42.675391 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:42.675409 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:42.675453 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:42.679019 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.679875 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.679884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.679887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.680034 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:42.680463 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:42.680468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:42.680470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:42.680882 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:42.681228 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:42.681579 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 22:47:42.681822 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 22:47:42.683084 6142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:42.727444 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.726834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58760)
0807 22:47:42.727112 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.727241 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 22:47:42.728830 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:42.728285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33065 (local address 127.0.0.1:58762)
0807 22:47:42.728540 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:42.728633 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260807 22:47:42.781332 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6010
I20260807 22:47:42.803381 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 5613
I20260807 22:47:42.804910 5685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:47:42.926781 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 5613
I20260807 22:48:02.953857 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--webserver_interface=127.20.10.190
--webserver_port=34397
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:45979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:03.340121 6204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:03.340464 6204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:03.340641 6204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:03.348621 6204 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:48:03.348749 6204 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:48:03.348840 6204 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:48:03.348943 6204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:03.349045 6204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:03.349172 6204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:03.349256 6204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:03.349344 6204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:03.360267 6204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:45979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:45979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=34397
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:03.361846 6204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:03.363484 6204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:03.378015 6210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:03.377974 6211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:03.378063 6213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:03.378614 6204 server_base.cc:1034] running on GCE node
I20260807 22:48:03.379675 6204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:03.381512 6204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:03.382884 6204 hybrid_clock.cc:648] HybridClock initialized: now 1786142883382746 us; error 73 us; skew 500 ppm
I20260807 22:48:03.383399 6204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:03 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142883, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:03.387843 6204 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:48:03.389863 6204 webserver.cc:460] Webserver started at http://127.20.10.190:34397/ using document root <none> and password file <none>
I20260807 22:48:03.390810 6204 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:03.390925 6204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:03.397969 6204 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 22:48:03.401674 6220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:03.403460 6204 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 22:48:03.403659 6204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "c7f726106e0444b3b69ea148c916b219"
format_stamp: "Formatted at 2026-08-07 22:47:16 on dist-test-slave-csg5"
server_key: "ccffa5cd5455a6aa4b04ea46d4a746ea"
server_key_iv: "edb07741aa428ef70c52432c74b54019"
server_key_version: "encryptionkey@0"
I20260807 22:48:03.404820 6204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:03.427558 6204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:03.433796 6204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:03.434087 6204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:03.459272 6204 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:45979
I20260807 22:48:03.459475 6274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:45979 every 8 connection(s)
I20260807 22:48:03.461268 6204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:03.462145 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6204
I20260807 22:48:03.463800 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:33065
--local_ip_for_outbound_sockets=127.20.10.129
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=45209
--webserver_interface=127.20.10.129
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:48:03.469864 6275 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:48:03.473647 6275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap starting.
I20260807 22:48:03.498144 6275 log.cc:824] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:03.516904 6275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:48:03.517621 6275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Bootstrap complete.
I20260807 22:48:03.527945 6275 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:48:03.529021 6275 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Initialized, Role: FOLLOWER
I20260807 22:48:03.530131 6275 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:48:03.530764 6275 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:03.530926 6275 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:03.531180 6275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:48:03.536891 6275 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:48:03.537554 6275 leader_election.cc:302] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f726106e0444b3b69ea148c916b219; no voters:
I20260807 22:48:03.538369 6275 leader_election.cc:288] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:48:03.538520 6279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:48:03.539979 6279 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [term 3 LEADER]: Becoming Leader. State: Replica: c7f726106e0444b3b69ea148c916b219, State: Running, Role: LEADER
I20260807 22:48:03.540750 6279 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } }
I20260807 22:48:03.541126 6275 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:03.543552 6281 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f726106e0444b3b69ea148c916b219. Latest consensus state: current_term: 3 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:48:03.543985 6281 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:03.544631 6280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c7f726106e0444b3b69ea148c916b219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f726106e0444b3b69ea148c916b219" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 45979 } } }
I20260807 22:48:03.544992 6280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:03.550722 6285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:03.563027 6285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:03.565589 6285 catalog_manager.cc:1259] Loaded cluster ID: e027ea0912a34df6b134f54a7e445c6e
I20260807 22:48:03.565876 6285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:03.571949 6285 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:03.574101 6285 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Loaded TSK: 0
I20260807 22:48:03.574417 6285 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7f726106e0444b3b69ea148c916b219: Loaded TSK: 1
F20260807 22:48:03.575831 6285 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f58d9c7a9ad google::LogMessage::Fail() at ??:0
@ 0x7f58d9c7d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f58d9c7a4de google::LogMessage::Flush() at ??:0
@ 0x7f58d9c7ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f58dd9de7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f58dd9cd73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f58ef6a91c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f58ef6a8783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f58ef7342a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f58ef7340c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f58ef77e5e9 std::function<>::operator()() at ??:0
@ 0x7f58ef6accdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f58ef6aac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f58ef732610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f58ef732439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f58dc8c6345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f58dc8c991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f58dc8c9799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f58dc8a4eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f58dfb7e6db start_thread at ??:0
@ 0x7f58d8b1461f clone at ??:0
W20260807 22:48:03.957063 6277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:03.957422 6277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:03.957537 6277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:03.966197 6277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:03.966364 6277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:03.966498 6277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:03.978274 6277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:33065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=45209
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:03.979810 6277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:03.981186 6277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:03.996061 6324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:03.996109 6323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:03.996132 6328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:03.997072 6277 server_base.cc:1034] running on GCE node
I20260807 22:48:03.997848 6277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:03.999130 6277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:04.000433 6277 hybrid_clock.cc:648] HybridClock initialized: now 1786142884000389 us; error 69 us; skew 500 ppm
I20260807 22:48:04.000926 6277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:04 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142884, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:04.005033 6277 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:48:04.006963 6277 webserver.cc:460] Webserver started at http://127.20.10.129:45209/ using document root <none> and password file <none>
I20260807 22:48:04.007849 6277 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:04.008014 6277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:04.015612 6277 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 22:48:04.019275 6335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:04.021080 6277 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:48:04.021267 6277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "c7ec7df9883a40ca99f1534a89befd0d"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "f38984ff0faf8e58ff99b4cbff8741cb"
server_key_iv: "938f6edb2778eefae366a8de5c61ba6a"
server_key_version: "encryptionkey@0"
I20260807 22:48:04.022447 6277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:04.044446 6277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:04.050351 6277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:04.050920 6277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:04.052433 6277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:04.054880 6342 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 22:48:04.061348 6342 ts_tablet_manager.cc:1852] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 22:48:04.062449 6342 ts_tablet_manager.cc:1905] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:04.071099 6342 ts_tablet_manager.cc:1918] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:04.071480 6342 ts_tablet_manager.cc:1939] T 05a969cbfd34461692160224529e160c P c7ec7df9883a40ca99f1534a89befd0d: Deleting consensus metadata
I20260807 22:48:04.071997 6277 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 22:48:04.072161 6277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260807 22:48:04.072345 6277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:04.072461 6277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:04.114553 6277 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:33065
I20260807 22:48:04.114605 6453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:33065 every 8 connection(s)
I20260807 22:48:04.116235 6277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:04.122238 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.120041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:56499)
0807 22:48:04.120945 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.121234 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":361,"threads_started":1}
W20260807 22:48:04.122928 6454 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:45979 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:45979: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
I20260807 22:48:04.124009 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.123549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:38889)
0807 22:48:04.123804 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.123888 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 22:48:04.125068 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.124672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:55069)
0807 22:48:04.124844 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.124931 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 22:48:04.125566 6454 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:48:04.125919 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6277
I20260807 22:48:04.126739 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:35813
--local_ip_for_outbound_sockets=127.20.10.130
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=36551
--webserver_interface=127.20.10.130
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:04.524763 6458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:04.525117 6458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:04.525233 6458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:04.533371 6458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:04.533507 6458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:04.533627 6458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:04.544649 6458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:35813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=36551
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:04.546195 6458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:04.547977 6458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:04.564214 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:04.564131 6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:04.564180 6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:04.565218 6458 server_base.cc:1034] running on GCE node
I20260807 22:48:04.566063 6458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:04.567378 6458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:04.568652 6458 hybrid_clock.cc:648] HybridClock initialized: now 1786142884568598 us; error 72 us; skew 500 ppm
I20260807 22:48:04.569046 6458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:04 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142884, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:04.573257 6458 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:48:04.575201 6458 webserver.cc:460] Webserver started at http://127.20.10.130:36551/ using document root <none> and password file <none>
I20260807 22:48:04.576012 6458 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:04.576144 6458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:04.582713 6458 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 22:48:04.586562 6475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:04.588478 6458 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:48:04.588675 6458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "0b72b6f875ce43efb74b3ad442eb5570"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-csg5"
server_key: "5fe4e161cc580a9354018c004cb9460e"
server_key_iv: "a5b25da6e73dba1d74743565c2d14170"
server_key_version: "encryptionkey@0"
I20260807 22:48:04.589846 6458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:04.605392 6458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:04.610589 6458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:04.610986 6458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:04.612226 6458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:04.614091 6458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:04.614217 6458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:04.614307 6458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:04.614415 6458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:04.655298 6458 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:35813
I20260807 22:48:04.655401 6596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:35813 every 8 connection(s)
I20260807 22:48:04.656893 6458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:04.662500 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.660417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:46519)
0807 22:48:04.661274 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.661558 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":323,"threads_started":1}
W20260807 22:48:04.663270 6597 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:45979 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:45979: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
I20260807 22:48:04.664252 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.663880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:35639)
0807 22:48:04.664052 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.664140 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:48:04.664913 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6458
I20260807 22:48:04.665344 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:04.664893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:58611)
0807 22:48:04.665042 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:04.665144 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:48:04.665596 6597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:48:04.665812 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:33537
--local_ip_for_outbound_sockets=127.20.10.131
--tserver_master_addrs=127.20.10.190:45979
--webserver_port=35761
--webserver_interface=127.20.10.131
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:05.065204 6600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:05.065541 6600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:05.065644 6600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:05.073511 6600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:05.073652 6600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:05.073794 6600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:05.084681 6600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:33537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=35761
--tserver_master_addrs=127.20.10.190:45979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:05.086483 6600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:05.087944 6600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:05.103425 6606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:05.103667 6608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:05.103531 6605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:05.104373 6600 server_base.cc:1034] running on GCE node
I20260807 22:48:05.105111 6600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:05.106195 6600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:05.107519 6600 hybrid_clock.cc:648] HybridClock initialized: now 1786142885107480 us; error 40 us; skew 500 ppm
I20260807 22:48:05.107964 6600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:05 dist-test-slave-csg5 krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142885, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:05.111706 6600 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:48:05.113574 6600 webserver.cc:460] Webserver started at http://127.20.10.131:35761/ using document root <none> and password file <none>
I20260807 22:48:05.114447 6600 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:05.114640 6600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:05.121289 6600 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260807 22:48:05.124790 6616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:05.126881 6600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:48:05.127460 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:05.126354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:45289)
0807 22:48:05.126983 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:05.127103 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":354,"threads_started":1}
I20260807 22:48:05.127091 6600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "257188db28464f6cb64ce736f6fadcf6"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-csg5"
server_key: "99d16a042a0c9b9318dc2a29e6a0c84c"
server_key_iv: "50b31ea141b63a1453cade7e7adf4a05"
server_key_version: "encryptionkey@0"
I20260807 22:48:05.128394 6600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:05.151674 6600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:05.157205 6600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:05.157636 6600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:05.159104 6600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:05.161128 6600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:05.161283 6600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:05.161434 6600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:05.161547 6600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:05.202775 6600 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:33537
I20260807 22:48:05.202963 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:33537 every 8 connection(s)
I20260807 22:48:05.204694 6600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:05.210531 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:05.208253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:47445)
0807 22:48:05.209149 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:05.209537 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":344,"threads_started":1}
W20260807 22:48:05.211542 6738 heartbeater.cc:644] Failed to heartbeat to 127.20.10.190:45979 (0 consecutive failures): Network error: Failed to ping master at 127.20.10.190:45979: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
I20260807 22:48:05.212641 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:05.212181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:51597)
0807 22:48:05.212426 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:05.212509 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 22:48:05.213917 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:05.213530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:39143)
0807 22:48:05.213695 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:05.213781 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 22:48:05.214463 6738 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:48:05.214998 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6600
I20260807 22:48:05.667550 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:05.666400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:53749)
0807 22:48:05.667118 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:05.667253 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":429,"threads_started":1}
I20260807 22:48:06.129519 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:06.128594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:37719)
0807 22:48:06.129173 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:06.129296 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":321,"threads_started":1}
I20260807 22:48:06.216398 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:06.215225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:43501)
0807 22:48:06.215991 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:06.216115 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":350,"threads_started":1}
I20260807 22:48:06.669811 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:06.668868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:59793)
0807 22:48:06.669488 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:06.669577 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":266,"threads_started":1}
I20260807 22:48:07.131807 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:07.130795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:50463)
0807 22:48:07.131381 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:07.131555 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":313,"threads_started":1}
I20260807 22:48:07.218405 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:07.217571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:32823)
0807 22:48:07.218110 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:07.218216 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":270,"threads_started":1}
I20260807 22:48:07.671895 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:07.670984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:59679)
0807 22:48:07.671554 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:07.671694 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":314,"threads_started":1}
I20260807 22:48:08.133882 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:08.132896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:35385)
0807 22:48:08.133531 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:08.133678 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":357,"threads_started":1}
I20260807 22:48:08.220963 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:08.219879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:32789)
0807 22:48:08.220513 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:08.220725 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":319,"threads_started":1}
I20260807 22:48:08.674052 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:08.673108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:42253)
0807 22:48:08.673746 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:08.673849 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":352,"threads_started":1}
I20260807 22:48:09.136250 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:09.135095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:47899)
0807 22:48:09.135816 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:09.135959 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":350,"threads_started":1}
I20260807 22:48:09.223415 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:09.222403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:43289)
0807 22:48:09.223116 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:09.223220 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":358,"threads_started":1}
I20260807 22:48:09.676177 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:09.675343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:51059)
0807 22:48:09.675887 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:09.675974 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":277,"threads_started":1}
I20260807 22:48:10.138767 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:10.137726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:40331)
0807 22:48:10.138345 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:10.138456 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":344,"threads_started":1}
I20260807 22:48:10.225558 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:10.224565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:43583)
0807 22:48:10.225169 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:10.225266 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":303,"threads_started":1}
I20260807 22:48:10.678165 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:10.677310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:51243)
0807 22:48:10.677866 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:10.677971 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":266,"threads_started":1}
I20260807 22:48:11.141325 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:11.140212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:37057)
0807 22:48:11.140873 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:11.140975 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":300,"threads_started":1}
I20260807 22:48:11.227619 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:11.226638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:54391)
0807 22:48:11.227316 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:11.227424 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":325,"threads_started":1}
I20260807 22:48:11.680629 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:11.679606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:41781)
0807 22:48:11.680218 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:11.680319 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260807 22:48:12.143582 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:12.142519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:32933)
0807 22:48:12.143266 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:12.143372 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":415,"threads_started":1}
I20260807 22:48:12.229830 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:12.228943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:56399)
0807 22:48:12.229503 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:12.229594 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":250,"threads_started":1}
I20260807 22:48:12.682914 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:12.681893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:57981)
0807 22:48:12.682546 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:12.682714 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":404,"threads_started":1}
I20260807 22:48:13.145493 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:13.144636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:54029)
0807 22:48:13.145212 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:13.145321 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":273,"threads_started":1}
I20260807 22:48:13.232384 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:13.231394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:41999)
0807 22:48:13.232009 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:13.232111 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":362,"threads_started":1}
I20260807 22:48:13.685318 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:13.684205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:52103)
0807 22:48:13.684918 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:13.685056 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":386,"threads_started":1}
I20260807 22:48:14.147689 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:14.146843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:60625)
0807 22:48:14.147400 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:14.147489 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":335,"threads_started":1}
I20260807 22:48:14.234916 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:14.233853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:34047)
0807 22:48:14.234474 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:14.234576 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":303,"threads_started":1}
I20260807 22:48:14.687637 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:14.686612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:55641)
0807 22:48:14.687334 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:14.687434 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":379,"threads_started":1}
I20260807 22:48:15.150224 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:15.149140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:46703)
0807 22:48:15.149825 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:15.149939 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":410,"threads_started":1}
I20260807 22:48:15.236992 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:15.236190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:54417)
0807 22:48:15.236715 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:15.236808 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":287,"threads_started":1}
I20260807 22:48:15.690043 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:15.689062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:32819)
0807 22:48:15.689708 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:15.689818 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":392,"threads_started":1}
I20260807 22:48:16.152631 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:16.151495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:48013)
0807 22:48:16.152035 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:16.152138 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260807 22:48:16.239181 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:16.238157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:37635)
0807 22:48:16.238850 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:16.238961 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":314,"threads_started":1}
I20260807 22:48:16.692389 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:16.691549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:38953)
0807 22:48:16.692091 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:16.692201 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":281,"threads_started":1}
I20260807 22:48:17.154891 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:17.153866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:50595)
0807 22:48:17.154483 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:17.154599 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":343,"threads_started":1}
I20260807 22:48:17.241397 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:17.240441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:57457)
0807 22:48:17.241077 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:17.241182 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":321,"threads_started":1}
I20260807 22:48:17.694550 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:17.693596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:43683)
0807 22:48:17.694227 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:17.694339 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":352,"threads_started":1}
I20260807 22:48:18.156934 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:18.155958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:42369)
0807 22:48:18.156561 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:18.156703 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":323,"threads_started":1}
I20260807 22:48:18.243712 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:18.242626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:54041)
0807 22:48:18.243372 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:18.243487 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":295,"threads_started":1}
I20260807 22:48:18.696553 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:18.695585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:56227)
0807 22:48:18.696207 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:18.696331 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":272,"threads_started":1}
I20260807 22:48:19.159157 6787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:19.158227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:34955)
0807 22:48:19.158871 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:19.158976 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":280,"threads_started":1}
I20260807 22:48:19.245669 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:19.244838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:55383)
0807 22:48:19.245361 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:19.245462 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":270,"threads_started":1}
I20260807 22:48:19.698917 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:19.697978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.130:44115)
0807 22:48:19.698517 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:19.698616 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":303,"threads_started":1}
I20260807 22:48:20.161201 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:20.160292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.129:50317)
0807 22:48:20.160882 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:20.160989 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":290,"threads_started":1}
I20260807 22:48:20.226306 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6277
I20260807 22:48:20.248080 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:20.247190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:45979 (local address 127.20.10.131:56241)
0807 22:48:20.247784 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:20.247884 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.190:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":323,"threads_started":1}
I20260807 22:48:20.254801 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:20.227771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.20.10.129:33065)
0807 22:48:20.228603 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:20.228626 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:20.228685 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:48:20.230628 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:20.230638 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:20.231281 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:20.231634 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:20.233508 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.235074 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.236557 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.237313 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.237476 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:20.245697 (+ 8221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:20.245711 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:20.245732 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:20.245791 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:20.249994 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.251176 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.251186 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.251193 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.251527 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.252110 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.252117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.252120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.252893 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:20.253198 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:20.253854 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:48:20.254394 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":396,"threads_started":1}
I20260807 22:48:20.256445 6449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:20.351274 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6277
I20260807 22:48:20.377465 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6458
I20260807 22:48:20.401291 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:20.378891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.20.10.130:35813)
0807 22:48:20.379396 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:20.379412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:20.379450 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:48:20.381070 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:20.381081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:20.381698 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:20.382111 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:20.384656 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.386583 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.388791 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.389669 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.389918 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:20.393014 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:20.393023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:20.393042 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:20.393101 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:20.396961 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.398002 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.398010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.398018 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.398244 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.398762 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.398766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.398768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.399439 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:20.399874 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:20.400433 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 22:48:20.400932 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":219,"threads_started":1}
I20260807 22:48:20.402972 6592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:20.498549 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6458
I20260807 22:48:20.524715 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6600
I20260807 22:48:20.548782 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:20.526076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.20.10.131:33537)
0807 22:48:20.526905 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:20.526920 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:20.526970 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:48:20.529185 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:20.529199 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:20.529731 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:20.530125 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:20.532257 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.533874 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.535878 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:20.536768 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:20.536943 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:20.540502 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:20.540512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:20.540532 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:20.540589 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:20.544113 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.545249 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.545258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.545271 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.545494 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:20.546003 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:20.546007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:20.546009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:20.546888 (+ 879us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:20.547173 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:20.547829 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:48:20.548383 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":347,"threads_started":1}
I20260807 22:48:20.550958 6733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:20.641222 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6600
2026-08-07T22:48:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66087 ms)
[----------] 30 tests from SecurityITest (340344 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:20 dist-test-slave-csg5 krb5kdc[6801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:20 dist-test-slave-csg5 krb5kdc[6801](info): set up 2 sockets
Aug 07 22:48:20 dist-test-slave-csg5 krb5kdc[6801](info): commencing operation
krb5kdc: starting...
W20260807 22:48:21.914829 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:22 dist-test-slave-csg5 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:48:22.300196 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33443
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:22.680605 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:22.680991 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:22.681173 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:22.690079 6815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:22.690313 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:22.690407 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:22.690505 6815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:22.690578 6815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:22.701236 6815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:22.702716 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:22.704136 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:22.718441 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:22.718947 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:22.718468 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:22.719645 6815 server_base.cc:1034] running on GCE node
I20260807 22:48:22.720523 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:22.722195 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:22.723484 6815 hybrid_clock.cc:648] HybridClock initialized: now 1786142902723426 us; error 53 us; skew 500 ppm
I20260807 22:48:22.723915 6815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:22 dist-test-slave-csg5 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:22.727588 6815 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:48:22.729336 6815 webserver.cc:460] Webserver started at http://127.20.10.190:36613/ using document root <none> and password file <none>
I20260807 22:48:22.730091 6815 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:22.730216 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:22.730496 6815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:22.733942 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "f9c4626894c44168bee873ee572241f5"
format_stamp: "Formatted at 2026-08-07 22:48:22 on dist-test-slave-csg5"
server_key: "f90f640546febb58ae6a3bca08773784"
server_key_iv: "9e2c636b9e98e0baaed2b61b883c1cad"
server_key_version: "encryptionkey@0"
I20260807 22:48:22.734870 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "f9c4626894c44168bee873ee572241f5"
format_stamp: "Formatted at 2026-08-07 22:48:22 on dist-test-slave-csg5"
server_key: "f90f640546febb58ae6a3bca08773784"
server_key_iv: "9e2c636b9e98e0baaed2b61b883c1cad"
server_key_version: "encryptionkey@0"
I20260807 22:48:22.740971 6815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:48:22.745201 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:22.746906 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:48:22.747139 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "f9c4626894c44168bee873ee572241f5"
format_stamp: "Formatted at 2026-08-07 22:48:22 on dist-test-slave-csg5"
server_key: "f90f640546febb58ae6a3bca08773784"
server_key_iv: "9e2c636b9e98e0baaed2b61b883c1cad"
server_key_version: "encryptionkey@0"
I20260807 22:48:22.747386 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:22.765496 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:22.770715 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:22.771047 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:22.793612 6815 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33443
I20260807 22:48:22.793613 6885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33443 every 8 connection(s)
I20260807 22:48:22.795240 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:22.798455 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6815
I20260807 22:48:22.798784 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:22.799654 20522 external_mini_cluster.cc:1428] Setting key d3254e2f6cd49172844011e0225d1dae
I20260807 22:48:22.800380 6886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:48:22 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:22.818773 6886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: Bootstrap starting.
I20260807 22:48:22.823102 6886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:22.824470 6886 log.cc:824] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:22.828135 6886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: No bootstrap required, opened a new log
I20260807 22:48:22.829514 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:22.802429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.20.10.190:33443)
0807 22:48:22.803395 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:22.803407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:22.803451 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:48:22.806085 (+ 2634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:22.806111 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:22.806577 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:22.807052 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:22.809326 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:22.810880 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:22.813124 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:22.813983 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:22.814155 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:22.820150 (+ 5995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:22.820160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:22.820183 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:22.820235 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:22.824336 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:22.825390 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:22.825399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:22.825406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:22.825624 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:22.826109 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:22.826115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:22.826118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:22.826862 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:22.827234 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:22.827750 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 22:48:22.828310 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":499,"threads_started":1}
I20260807 22:48:22.837832 6886 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } }
I20260807 22:48:22.838414 6886 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:22.838563 6886 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c4626894c44168bee873ee572241f5, State: Initialized, Role: FOLLOWER
I20260807 22:48:22.839270 6886 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } }
I20260807 22:48:22.839538 6886 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:22.839717 6886 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:22.839915 6886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:22.842965 6886 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } }
I20260807 22:48:22.843592 6886 leader_election.cc:302] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9c4626894c44168bee873ee572241f5; no voters:
I20260807 22:48:22.844333 6886 leader_election.cc:288] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:22.844636 6891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:22.845498 6891 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [term 1 LEADER]: Becoming Leader. State: Replica: f9c4626894c44168bee873ee572241f5, State: Running, Role: LEADER
I20260807 22:48:22.846155 6891 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } }
I20260807 22:48:22.846781 6886 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:22.850186 6893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9c4626894c44168bee873ee572241f5. Latest consensus state: current_term: 1 leader_uuid: "f9c4626894c44168bee873ee572241f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } } }
I20260807 22:48:22.850878 6893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:22.850885 6892 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9c4626894c44168bee873ee572241f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c4626894c44168bee873ee572241f5" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33443 } } }
I20260807 22:48:22.851300 6892 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:22.854121 6899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:22.861573 6899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:22.871950 6899 catalog_manager.cc:1347] Generated new cluster ID: a613854fc68b4d9e90fe7fb1ec9ab870
I20260807 22:48:22.872102 6899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:22.882521 6899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:22.884033 6899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:22.895165 6899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: Generated new TSK 0
I20260807 22:48:22.896232 6899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:48:22.995985 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:23.394855 6913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:23.395394 6913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:23.395618 6913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:23.405313 6913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:23.405474 6913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:23.405603 6913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:23.417205 6913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:23.418740 6913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:23.420234 6913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:23.437783 6920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:23.437917 6922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:23.437983 6919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:23.439036 6913 server_base.cc:1034] running on GCE node
I20260807 22:48:23.439756 6913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:23.440930 6913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:23.442173 6913 hybrid_clock.cc:648] HybridClock initialized: now 1786142903442116 us; error 47 us; skew 500 ppm
I20260807 22:48:23.442520 6913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:23 dist-test-slave-csg5 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142903, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:23.446064 6913 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:48:23.447956 6913 webserver.cc:460] Webserver started at http://127.20.10.129:45265/ using document root <none> and password file <none>
I20260807 22:48:23.448701 6913 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:23.448827 6913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:23.449129 6913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:23.452837 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "eb8f957d23c24f6995dcd1da2f0345b5"
format_stamp: "Formatted at 2026-08-07 22:48:23 on dist-test-slave-csg5"
server_key: "238a73f6f7a73d1e4a48efe08b88e4ad"
server_key_iv: "c5a4042b846496c9e4fcd7ad4fbdb2a4"
server_key_version: "encryptionkey@0"
I20260807 22:48:23.453840 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "eb8f957d23c24f6995dcd1da2f0345b5"
format_stamp: "Formatted at 2026-08-07 22:48:23 on dist-test-slave-csg5"
server_key: "238a73f6f7a73d1e4a48efe08b88e4ad"
server_key_iv: "c5a4042b846496c9e4fcd7ad4fbdb2a4"
server_key_version: "encryptionkey@0"
I20260807 22:48:23.460013 6913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:48:23.464092 6929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:23.466063 6913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:48:23.466249 6913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "eb8f957d23c24f6995dcd1da2f0345b5"
format_stamp: "Formatted at 2026-08-07 22:48:23 on dist-test-slave-csg5"
server_key: "238a73f6f7a73d1e4a48efe08b88e4ad"
server_key_iv: "c5a4042b846496c9e4fcd7ad4fbdb2a4"
server_key_version: "encryptionkey@0"
I20260807 22:48:23.466452 6913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:23.484915 6913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:23.490633 6913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:23.491009 6913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:23.492159 6913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:23.493819 6913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:23.493909 6913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:23.494017 6913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:23.494071 6913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:23.535182 6913 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:36033
I20260807 22:48:23.535221 7048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:36033 every 8 connection(s)
I20260807 22:48:23.536868 6913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:23.547034 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 6913
I20260807 22:48:23.547292 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:23.548080 20522 external_mini_cluster.cc:1428] Setting key 09a059dcdd8d17346062c5caa1a2ce87
Aug 07 22:48:23 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142903, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:23.563503 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:23.539947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:51225 (local address 127.20.10.190:33443)
0807 22:48:23.540491 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:23.540496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:23.541351 (+ 855us) server_negotiation.cc:407] Connection header received
0807 22:48:23.543722 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:23.543730 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:23.543844 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:23.543985 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:23.545825 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:23.546830 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:23.548979 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:23.549628 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:23.549783 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:23.554217 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:23.554227 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:23.554235 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:23.554293 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:23.558760 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:23.559859 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:23.559869 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:23.559875 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:23.560066 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:23.560773 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:23.560781 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:23.560788 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:23.561246 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:23.561422 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:23.562444 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0807 22:48:23.562797 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":309,"threads_started":1}
I20260807 22:48:23.564256 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:23.540313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33443 (local address 127.20.10.129:51225)
0807 22:48:23.541153 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:23.541200 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:23.543382 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:23.544289 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:23.544299 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:23.544923 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:23.545605 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:23.545624 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:23.547026 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:23.547030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:23.548793 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:23.548809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:23.549934 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:23.549941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:23.550264 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:23.551100 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:23.551124 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:23.553884 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:23.558999 (+ 5115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:23.559008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:23.559025 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:23.559693 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:23.560372 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:23.560376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:23.560379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:23.560627 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:23.561429 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:23.561436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:23.561884 (+ 448us) client_negotiation.cc:769] Sending connection context
0807 22:48:23.562299 (+ 415us) client_negotiation.cc:240] Negotiation successful
0807 22:48:23.562898 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":339,"threads_started":1}
I20260807 22:48:23.566351 7049 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33443
I20260807 22:48:23.566807 7049 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:23.567744 7049 heartbeater.cc:511] Master 127.20.10.190:33443 requested a full tablet report, sending...
I20260807 22:48:23.570286 6848 ts_manager.cc:194] Registered new tserver with Master: eb8f957d23c24f6995dcd1da2f0345b5 (127.20.10.129:36033)
I20260807 22:48:23.572167 6848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:51225
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:48:23.631999 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:24.009567 7055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:24.009900 7055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:24.010079 7055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:24.017738 7055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:24.017870 7055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:24.018064 7055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:24.028534 7055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:24.030042 7055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:24.031615 7055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:24.046629 7061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:24.046693 7064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:24.046718 7062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:24.048671 7055 server_base.cc:1034] running on GCE node
I20260807 22:48:24.049427 7055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:24.050503 7055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:24.051726 7055 hybrid_clock.cc:648] HybridClock initialized: now 1786142904051665 us; error 83 us; skew 500 ppm
I20260807 22:48:24.052130 7055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:24 dist-test-slave-csg5 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:24.056097 7055 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:48:24.057894 7055 webserver.cc:460] Webserver started at http://127.20.10.130:44201/ using document root <none> and password file <none>
I20260807 22:48:24.058629 7055 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:24.058835 7055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:24.059146 7055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:24.062338 7055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "1702827247bf4ca6893e245643a699df"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "5a64b7a363e14d4d36c9acc9111b8d7e"
server_key_iv: "b826781bda7d4945135b688c524c7a2a"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.063279 7055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "1702827247bf4ca6893e245643a699df"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "5a64b7a363e14d4d36c9acc9111b8d7e"
server_key_iv: "b826781bda7d4945135b688c524c7a2a"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.069756 7055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:48:24.074059 7072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.076186 7055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:48:24.076519 7055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "1702827247bf4ca6893e245643a699df"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "5a64b7a363e14d4d36c9acc9111b8d7e"
server_key_iv: "b826781bda7d4945135b688c524c7a2a"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.076786 7055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:24.103709 7055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:24.109591 7055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:24.109982 7055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:24.111302 7055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:24.112998 7055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:24.113147 7055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.113329 7055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:24.113415 7055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.156428 7055 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:37827
I20260807 22:48:24.156507 7195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:37827 every 8 connection(s)
I20260807 22:48:24.158032 7055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:24.160657 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7055
I20260807 22:48:24.160869 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:24.161414 20522 external_mini_cluster.cc:1428] Setting key 704e9d8949cb67671ce386e33b31a754
Aug 07 22:48:24 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:24.187393 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.161329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49231 (local address 127.20.10.190:33443)
0807 22:48:24.164538 (+ 3209us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.164543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.164564 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:48:24.166892 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.166897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.166978 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.167088 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:24.169510 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.170532 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.173269 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.173785 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.173903 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.179216 (+ 5313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:24.179225 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:24.179229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:24.179272 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:24.183491 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.184454 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.184462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.184469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.184655 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.185188 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.185196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.185203 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.185629 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:24.185814 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.186556 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.186881 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3082,"thread_start_us":2956,"threads_started":1}
I20260807 22:48:24.188197 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.162302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33443 (local address 127.20.10.130:49231)
0807 22:48:24.163487 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:24.163555 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:24.166467 (+ 2912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:24.167419 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:24.167435 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:24.168324 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:24.169247 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.169279 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.170769 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.170777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.173061 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.173083 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.174074 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.174083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.174442 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.175335 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:24.175378 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:24.178766 (+ 3388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:24.183710 (+ 4944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:24.183718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:24.183741 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:24.184285 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:24.184827 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:24.184830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:24.184832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:24.185046 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:24.185778 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:24.185784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:24.186148 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 22:48:24.186405 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 22:48:24.186969 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":493,"threads_started":1}
I20260807 22:48:24.189893 7196 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33443
I20260807 22:48:24.190331 7196 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:24.191210 7196 heartbeater.cc:511] Master 127.20.10.190:33443 requested a full tablet report, sending...
I20260807 22:48:24.193058 6848 ts_manager.cc:194] Registered new tserver with Master: 1702827247bf4ca6893e245643a699df (127.20.10.130:37827)
I20260807 22:48:24.194036 6848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:49231
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:48:24.251044 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:48:24.576004 7049 heartbeater.cc:503] Master 127.20.10.190:33443 was elected leader, sending a full tablet report...
W20260807 22:48:24.640152 7203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:24.640609 7203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:24.640798 7203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:24.649034 7203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:24.649219 7203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:24.649423 7203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:24.660511 7203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:24.661968 7203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:24.663467 7203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:24.679226 7210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:24.679193 7212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:24.679464 7209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:24.680351 7203 server_base.cc:1034] running on GCE node
I20260807 22:48:24.681386 7203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:24.682708 7203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:24.684013 7203 hybrid_clock.cc:648] HybridClock initialized: now 1786142904683957 us; error 40 us; skew 500 ppm
I20260807 22:48:24.684465 7203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:24 dist-test-slave-csg5 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:24.688612 7203 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:48:24.690454 7203 webserver.cc:460] Webserver started at http://127.20.10.131:44335/ using document root <none> and password file <none>
I20260807 22:48:24.691313 7203 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:24.691438 7203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:24.691728 7203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:24.695226 7203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "23924304fdbe45d690e30b76b14e8326"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "8e64e28aa85aeab42cf27388f276cb3a"
server_key_iv: "22a1c7ed74d3ac9b124b62b10f4a6b5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.696080 7203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "23924304fdbe45d690e30b76b14e8326"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "8e64e28aa85aeab42cf27388f276cb3a"
server_key_iv: "22a1c7ed74d3ac9b124b62b10f4a6b5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.702241 7203 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:48:24.707221 7220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.709118 7203 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:48:24.709313 7203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "23924304fdbe45d690e30b76b14e8326"
format_stamp: "Formatted at 2026-08-07 22:48:24 on dist-test-slave-csg5"
server_key: "8e64e28aa85aeab42cf27388f276cb3a"
server_key_iv: "22a1c7ed74d3ac9b124b62b10f4a6b5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:24.709654 7203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:24.751129 7203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:24.756754 7203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:24.757055 7203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:24.758237 7203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:24.759930 7203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:24.760020 7203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.760133 7203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:24.760174 7203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:24.800035 7203 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:46707
I20260807 22:48:24.800141 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:46707 every 8 connection(s)
I20260807 22:48:24.801626 7203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:24.812039 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7203
I20260807 22:48:24.812260 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:24.812959 20522 external_mini_cluster.cc:1428] Setting key a44ec8a08270c09e06d859a2d85ce110
Aug 07 22:48:24 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:24.825067 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.804514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:58027 (local address 127.20.10.190:33443)
0807 22:48:24.805061 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.805066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.805986 (+ 920us) server_negotiation.cc:407] Connection header received
0807 22:48:24.808031 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.808036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.808141 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.808266 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:24.809863 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.810746 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.812475 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.813247 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.813404 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.817955 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:24.817961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:24.817965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:24.818005 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:24.821078 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.822199 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.822214 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.822217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.822447 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.823092 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.823097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.823099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.823548 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:24.823697 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.824529 (+ 832us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.824751 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":304,"threads_started":1}
I20260807 22:48:24.826330 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.805057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33443 (local address 127.20.10.131:58027)
0807 22:48:24.805824 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:24.805863 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:24.807733 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:24.808532 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:24.808540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:24.809094 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:24.809674 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.809693 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.810912 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.810916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.812257 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.812270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.813512 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.813517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.813840 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.814784 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:24.814815 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:24.817695 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:24.821383 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:24.821393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:24.821410 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:24.822012 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:24.822594 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:24.822598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:24.822601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:24.822935 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:24.823698 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:24.823707 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:24.824103 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 22:48:24.824393 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 22:48:24.824866 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":293,"threads_started":1}
I20260807 22:48:24.828081 7346 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33443
I20260807 22:48:24.828428 7346 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:24.829247 7346 heartbeater.cc:511] Master 127.20.10.190:33443 requested a full tablet report, sending...
I20260807 22:48:24.830943 6848 ts_manager.cc:194] Registered new tserver with Master: 23924304fdbe45d690e30b76b14e8326 (127.20.10.131:46707)
I20260807 22:48:24.831914 6848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:58027
I20260807 22:48:24.842941 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:24.865878 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.848990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.20.10.190:33443)
0807 22:48:24.849291 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.849301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.849785 (+ 484us) server_negotiation.cc:407] Connection header received
0807 22:48:24.850050 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.850062 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.850143 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.850274 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:24.852367 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.853411 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.855307 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.855781 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.855944 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.859116 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:24.859122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:24.859126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:24.859178 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:24.862382 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.863430 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.863444 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.863448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.863595 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:24.864072 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:24.864076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:24.864078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:24.864473 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:24.864773 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.865233 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.865500 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 22:48:24.871057 6848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:24.879452 6848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:24.915633 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.894995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.20.10.129:36033)
0807 22:48:24.906049 (+ 11054us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.906064 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.906114 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:48:24.906280 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.906294 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.906700 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.907005 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:24.907839 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.909489 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.912555 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.914267 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.914430 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.914745 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.915070 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.915269 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10757,"thread_start_us":12284,"threads_started":1}
I20260807 22:48:24.916086 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.894507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:36033 (local address 127.0.0.1:41764)
0807 22:48:24.896739 (+ 2232us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:24.896811 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:24.897062 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:24.907019 (+ 9957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:24.907027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:24.907063 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:24.907540 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.907566 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.909650 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.909658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.912374 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.912393 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.914492 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.914499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.914777 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.914850 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 22:48:24.915001 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:48:24.915279 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"mutex_wait_us":1,"thread_start_us":901,"threads_started":1}
I20260807 22:48:24.917092 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.895456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32892 (local address 127.20.10.130:37827)
0807 22:48:24.896082 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.896090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.908202 (+ 12112us) server_negotiation.cc:407] Connection header received
0807 22:48:24.908536 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.908545 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.908816 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.909004 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:24.910170 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.911827 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.913911 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.915608 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.915757 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.915975 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.916541 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.916722 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":302,"threads_started":1}
I20260807 22:48:24.917176 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.894875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:37827 (local address 127.0.0.1:32892)
0807 22:48:24.908047 (+ 13172us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:24.908078 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:24.908326 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:24.909210 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:24.909214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:24.909238 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:24.909975 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.909993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.911976 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.911981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.913739 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.913753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.916146 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.916155 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.916351 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.916376 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:48:24.916490 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:48:24.916635 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12993,"mutex_wait_us":66,"thread_start_us":12808,"threads_started":1}
I20260807 22:48:24.918310 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.895475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:46707 (local address 127.0.0.1:59024)
0807 22:48:24.908836 (+ 13361us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:24.908856 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:24.909007 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:24.909704 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:24.909711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:24.909747 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:24.910304 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.910322 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.912405 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.912413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.914694 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:24.914714 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.917580 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.917589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:24.917779 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.917814 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:48:24.917901 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 22:48:24.918021 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13274,"mutex_wait_us":58,"thread_start_us":13759,"threads_started":1}
I20260807 22:48:24.919340 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:24.895950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.20.10.131:46707)
0807 22:48:24.908607 (+ 12657us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:24.908618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:24.908969 (+ 351us) server_negotiation.cc:407] Connection header received
0807 22:48:24.909174 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:24.909184 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:24.909412 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:24.909687 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:24.910800 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.912261 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.914865 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:24.916954 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:24.917083 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:24.918858 (+ 1775us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:24.918922 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 22:48:24.919058 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12465,"thread_start_us":12287,"threads_started":1}
I20260807 22:48:24.921718 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce (DEFAULT_TABLE table=test-table [id=13156c4a770e46f39be7dad9c26cf4f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:24.921864 6979 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce (DEFAULT_TABLE table=test-table [id=13156c4a770e46f39be7dad9c26cf4f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:24.923403 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf2932c21b8411c8febdd8fbabeacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:24.923439 6979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf2932c21b8411c8febdd8fbabeacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:24.922933 7273 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce (DEFAULT_TABLE table=test-table [id=13156c4a770e46f39be7dad9c26cf4f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:24.924388 7273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf2932c21b8411c8febdd8fbabeacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:24.937237 7368 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Bootstrap starting.
I20260807 22:48:24.939685 7369 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Bootstrap starting.
I20260807 22:48:24.942705 7370 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Bootstrap starting.
I20260807 22:48:24.942749 7368 tablet_bootstrap.cc:654] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:24.943712 7369 tablet_bootstrap.cc:654] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:24.944111 7368 log.cc:824] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:24.945148 7369 log.cc:824] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:24.947422 7370 tablet_bootstrap.cc:654] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:24.948549 7369 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: No bootstrap required, opened a new log
I20260807 22:48:24.948681 7368 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: No bootstrap required, opened a new log
I20260807 22:48:24.948892 7369 ts_tablet_manager.cc:1397] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:48:24.948933 7370 log.cc:824] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:24.948992 7368 ts_tablet_manager.cc:1397] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260807 22:48:24.953258 7370 tablet_bootstrap.cc:492] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: No bootstrap required, opened a new log
I20260807 22:48:24.953576 7370 ts_tablet_manager.cc:1397] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260807 22:48:24.958187 7369 raft_consensus.cc:348] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.958633 7369 raft_consensus.cc:374] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:24.958773 7369 raft_consensus.cc:729] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1702827247bf4ca6893e245643a699df, State: Initialized, Role: FOLLOWER
I20260807 22:48:24.959487 7369 consensus_queue.cc:260] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.960759 7196 heartbeater.cc:503] Master 127.20.10.190:33443 was elected leader, sending a full tablet report...
I20260807 22:48:24.961361 7369 ts_tablet_manager.cc:1428] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 22:48:24.960227 7368 raft_consensus.cc:348] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.962571 7368 raft_consensus.cc:374] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:24.962711 7368 raft_consensus.cc:729] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb8f957d23c24f6995dcd1da2f0345b5, State: Initialized, Role: FOLLOWER
I20260807 22:48:24.963454 7368 consensus_queue.cc:260] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.965008 7370 raft_consensus.cc:348] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.965553 7368 ts_tablet_manager.cc:1428] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260807 22:48:24.965551 7370 raft_consensus.cc:374] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:24.965659 7370 raft_consensus.cc:729] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23924304fdbe45d690e30b76b14e8326, State: Initialized, Role: FOLLOWER
I20260807 22:48:24.966337 7370 consensus_queue.cc:260] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:24.971563 7346 heartbeater.cc:503] Master 127.20.10.190:33443 was elected leader, sending a full tablet report...
I20260807 22:48:24.972332 7370 ts_tablet_manager.cc:1428] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
W20260807 22:48:25.040199 7050 tablet.cc:2369] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:25.053725 7347 tablet.cc:2369] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:25.160954 7197 tablet.cc:2369] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:25.168076 7376 raft_consensus.cc:482] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:25.168385 7376 raft_consensus.cc:504] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:25.170087 7376 leader_election.cc:288] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1702827247bf4ca6893e245643a699df (127.20.10.130:37827), 23924304fdbe45d690e30b76b14e8326 (127.20.10.131:46707)
I20260807 22:48:25.178182 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.170872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:47733 (local address 127.20.10.131:46707)
0807 22:48:25.171125 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.171134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.171973 (+ 839us) server_negotiation.cc:407] Connection header received
0807 22:48:25.172198 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.172203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.172285 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.172398 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:25.173164 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.174026 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.176313 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.177205 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.177388 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.177466 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.177799 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.177942 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:48:25.178216 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.170521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:43873 (local address 127.20.10.130:37827)
0807 22:48:25.170799 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.170810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.171839 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 22:48:25.172080 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.172085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.172148 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.172262 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:25.172949 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.173824 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.176015 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.177064 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.177201 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.177265 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.177706 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.177881 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 22:48:25.178421 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.170468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:37827 (local address 127.20.10.129:43873)
0807 22:48:25.171679 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:25.171708 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:25.171924 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:25.172279 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:25.172283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:25.172306 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:25.172747 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:25.172757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.173971 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.173979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:25.175831 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:25.175846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.177325 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.177333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:25.177533 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.177559 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:48:25.177638 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:48:25.177800 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":49,"thread_start_us":569,"threads_started":1}
I20260807 22:48:25.178723 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.170621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:46707 (local address 127.20.10.129:47733)
0807 22:48:25.171842 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:25.171862 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:25.172046 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:25.172427 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:25.172432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:25.172475 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:25.172981 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:25.172996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.174195 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.174199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:25.176142 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:25.176159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.177460 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.177465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:25.177593 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.177651 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 22:48:25.177770 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:48:25.177906 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":49,"thread_start_us":872,"threads_started":1}
I20260807 22:48:25.179800 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf2932c21b8411c8febdd8fbabeacce" candidate_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1702827247bf4ca6893e245643a699df" is_pre_election: true
I20260807 22:48:25.179991 7294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf2932c21b8411c8febdd8fbabeacce" candidate_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23924304fdbe45d690e30b76b14e8326" is_pre_election: true
I20260807 22:48:25.180423 7144 raft_consensus.cc:2454] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb8f957d23c24f6995dcd1da2f0345b5 in term 0.
I20260807 22:48:25.180577 7294 raft_consensus.cc:2454] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb8f957d23c24f6995dcd1da2f0345b5 in term 0.
I20260807 22:48:25.181221 6930 leader_election.cc:302] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1702827247bf4ca6893e245643a699df, eb8f957d23c24f6995dcd1da2f0345b5; no voters:
I20260807 22:48:25.181814 7376 raft_consensus.cc:2781] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:25.182013 7376 raft_consensus.cc:482] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:25.182159 7376 raft_consensus.cc:3037] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:25.184630 7376 raft_consensus.cc:504] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:25.185380 7376 leader_election.cc:288] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [CANDIDATE]: Term 1 election: Requested vote from peers 1702827247bf4ca6893e245643a699df (127.20.10.130:37827), 23924304fdbe45d690e30b76b14e8326 (127.20.10.131:46707)
I20260807 22:48:25.186031 7294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf2932c21b8411c8febdd8fbabeacce" candidate_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23924304fdbe45d690e30b76b14e8326"
I20260807 22:48:25.186090 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf2932c21b8411c8febdd8fbabeacce" candidate_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1702827247bf4ca6893e245643a699df"
I20260807 22:48:25.186283 7294 raft_consensus.cc:3037] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:25.186488 7144 raft_consensus.cc:3037] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:25.188678 7294 raft_consensus.cc:2454] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb8f957d23c24f6995dcd1da2f0345b5 in term 1.
I20260807 22:48:25.189144 7144 raft_consensus.cc:2454] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb8f957d23c24f6995dcd1da2f0345b5 in term 1.
I20260807 22:48:25.189266 6932 leader_election.cc:302] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23924304fdbe45d690e30b76b14e8326, eb8f957d23c24f6995dcd1da2f0345b5; no voters:
I20260807 22:48:25.189687 7376 raft_consensus.cc:2781] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:25.190196 7376 raft_consensus.cc:686] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 1 LEADER]: Becoming Leader. State: Replica: eb8f957d23c24f6995dcd1da2f0345b5, State: Running, Role: LEADER
I20260807 22:48:25.190788 7376 consensus_queue.cc:237] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } }
I20260807 22:48:25.196420 6848 catalog_manager.cc:5495] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb8f957d23c24f6995dcd1da2f0345b5 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb8f957d23c24f6995dcd1da2f0345b5" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 } health_report { overall_health: UNKNOWN } } }
I20260807 22:48:25.216238 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.209285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.20.10.129:36033)
0807 22:48:25.209515 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.209520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.209725 (+ 205us) server_negotiation.cc:407] Connection header received
0807 22:48:25.209921 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.209925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.210039 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.210183 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:25.210863 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.211733 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.213376 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.213889 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.214017 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.214913 (+ 896us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:25.215586 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:25.215710 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.215911 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.215998 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:48:25.226011 7294 raft_consensus.cc:1261] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 1 FOLLOWER]: Refusing update from remote peer eb8f957d23c24f6995dcd1da2f0345b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:25.226011 7144 raft_consensus.cc:1261] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 1 FOLLOWER]: Refusing update from remote peer eb8f957d23c24f6995dcd1da2f0345b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:25.227442 7376 consensus_queue.cc:1035] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1702827247bf4ca6893e245643a699df" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:25.227824 7380 consensus_queue.cc:1035] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23924304fdbe45d690e30b76b14e8326" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:25.241214 7383 mvcc.cc:205] Tried to move back new op lower bound from 7316041339793776640 to 7316041339665534976. Current Snapshot: MvccSnapshot[applied={T|T < 7316041339793776640}]
I20260807 22:48:25.259104 6846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:25.259502 6846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:25.264384 6846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9c4626894c44168bee873ee572241f5: Sending DeleteTablet for 3 replicas of tablet 4bf2932c21b8411c8febdd8fbabeacce
I20260807 22:48:25.265936 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:25 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:32892
I20260807 22:48:25.265933 6979 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:25 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:41764
I20260807 22:48:25.266072 7273 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf2932c21b8411c8febdd8fbabeacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:25 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:59024
I20260807 22:48:25.272197 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6913
I20260807 22:48:25.278436 7395 tablet_replica.cc:330] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: stopping tablet replica
I20260807 22:48:25.278999 7395 raft_consensus.cc:2229] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:25.279984 7395 raft_consensus.cc:2258] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:25.281409 7394 tablet_replica.cc:330] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: stopping tablet replica
I20260807 22:48:25.282095 7394 raft_consensus.cc:2229] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:25.282739 7394 raft_consensus.cc:2258] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:25.283052 7395 ts_tablet_manager.cc:1905] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:25.284428 7393 tablet_replica.cc:330] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: stopping tablet replica
I20260807 22:48:25.284976 7393 raft_consensus.cc:2229] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:25.285615 7393 raft_consensus.cc:2258] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:25.288237 7393 ts_tablet_manager.cc:1905] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:25.291889 7395 ts_tablet_manager.cc:1918] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:25.292099 7395 log.cc:1196] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/4bf2932c21b8411c8febdd8fbabeacce
I20260807 22:48:25.292534 7394 ts_tablet_manager.cc:1905] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:25.292696 7395 ts_tablet_manager.cc:1939] T 4bf2932c21b8411c8febdd8fbabeacce P eb8f957d23c24f6995dcd1da2f0345b5: Deleting consensus metadata
I20260807 22:48:25.294807 6834 catalog_manager.cc:4841] TS eb8f957d23c24f6995dcd1da2f0345b5 (127.20.10.129:36033): tablet 4bf2932c21b8411c8febdd8fbabeacce (table test-table [id=13156c4a770e46f39be7dad9c26cf4f7]) successfully deleted
I20260807 22:48:25.295925 7393 ts_tablet_manager.cc:1918] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:25.296113 7393 log.cc:1196] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/4bf2932c21b8411c8febdd8fbabeacce
I20260807 22:48:25.296728 7393 ts_tablet_manager.cc:1939] T 4bf2932c21b8411c8febdd8fbabeacce P 1702827247bf4ca6893e245643a699df: Deleting consensus metadata
I20260807 22:48:25.298857 6832 catalog_manager.cc:4841] TS 1702827247bf4ca6893e245643a699df (127.20.10.130:37827): tablet 4bf2932c21b8411c8febdd8fbabeacce (table test-table [id=13156c4a770e46f39be7dad9c26cf4f7]) successfully deleted
I20260807 22:48:25.301453 7394 ts_tablet_manager.cc:1918] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:25.301677 7394 log.cc:1196] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/4bf2932c21b8411c8febdd8fbabeacce
I20260807 22:48:25.302263 7394 ts_tablet_manager.cc:1939] T 4bf2932c21b8411c8febdd8fbabeacce P 23924304fdbe45d690e30b76b14e8326: Deleting consensus metadata
I20260807 22:48:25.304512 6833 catalog_manager.cc:4841] TS 23924304fdbe45d690e30b76b14e8326 (127.20.10.131:46707): tablet 4bf2932c21b8411c8febdd8fbabeacce (table test-table [id=13156c4a770e46f39be7dad9c26cf4f7]) successfully deleted
Aug 07 22:48:25 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:48:25.315470 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.275894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.20.10.129:36033)
0807 22:48:25.276172 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.276179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.283680 (+ 7501us) server_negotiation.cc:407] Connection header received
0807 22:48:25.283899 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.283907 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.284014 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.284134 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:25.286908 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.288890 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.295247 (+ 6357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.296082 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.296201 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.306455 (+ 10254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:25.306482 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:25.306505 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:25.306571 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:25.311489 (+ 4918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.312452 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.312464 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.312469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.312703 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.313229 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.313238 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.313243 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.313895 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:25.314276 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.314619 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.315015 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:48:25.316459 7044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:25.427361 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6913
I20260807 22:48:25.458408 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7055
Aug 07 22:48:25 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:48:25.476723 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.459478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.20.10.130:37827)
0807 22:48:25.459663 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.459667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.459686 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:48:25.459804 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.459808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.459926 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.460031 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:25.461492 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.462853 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.464394 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.464823 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.464935 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.469826 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:25.469844 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:25.469861 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:25.469905 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:25.473032 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.473918 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.473928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.473931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.474105 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.474609 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.474614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.474616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.475416 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:25.475699 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.476044 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.476275 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:48:25.477432 7191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:25.577736 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7055
I20260807 22:48:25.603153 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7203
Aug 07 22:48:25 dist-test-slave-csg5 krb5kdc[6801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:48:25.621850 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:25.604163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.20.10.131:46707)
0807 22:48:25.604416 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:25.604421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:25.604442 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:48:25.604614 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:25.604619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:25.604695 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:25.604799 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:25.606372 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.607424 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.608950 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:25.609426 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:25.609538 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:25.615093 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:25.615113 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:25.615130 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:25.615171 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:25.618237 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.619136 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.619145 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.619147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.619306 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:25.619717 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:25.619722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:25.619727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:25.620372 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:25.620785 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:25.621141 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 22:48:25.621418 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260807 22:48:25.622735 7341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:25.720153 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7203
I20260807 22:48:25.743863 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 6815
I20260807 22:48:25.745312 6881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:25.866963 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 6815
2026-08-07T22:48:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5231 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T22:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:25Z Disabled control of system clock
I20260807 22:48:25.980556 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35097
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:26.370267 7404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:26.370710 7404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:26.370869 7404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:26.378599 7404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:26.378846 7404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:26.378931 7404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:26.379019 7404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:26.379108 7404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:26.390565 7404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:35097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:26.392177 7404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:26.394348 7404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:26.409675 7409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:26.409646 7410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:26.410351 7412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:26.411418 7404 server_base.cc:1034] running on GCE node
I20260807 22:48:26.412431 7404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:26.414047 7404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:26.415445 7404 hybrid_clock.cc:648] HybridClock initialized: now 1786142906415371 us; error 70 us; skew 500 ppm
I20260807 22:48:26.415879 7404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:26.419041 7404 webserver.cc:460] Webserver started at http://127.20.10.190:33945/ using document root <none> and password file <none>
I20260807 22:48:26.419929 7404 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:26.420083 7404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:26.420449 7404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:26.423954 7404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56"
format_stamp: "Formatted at 2026-08-07 22:48:26 on dist-test-slave-csg5"
server_key: "c6bdc1ab42947efd4dabf10b00029474"
server_key_iv: "8a8e6ea6a8b27c2e5ed2c0e291678f94"
server_key_version: "encryptionkey@0"
I20260807 22:48:26.424904 7404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56"
format_stamp: "Formatted at 2026-08-07 22:48:26 on dist-test-slave-csg5"
server_key: "c6bdc1ab42947efd4dabf10b00029474"
server_key_iv: "8a8e6ea6a8b27c2e5ed2c0e291678f94"
server_key_version: "encryptionkey@0"
I20260807 22:48:26.431124 7404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:48:26.435348 7418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:26.437062 7404 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:48:26.437381 7404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56"
format_stamp: "Formatted at 2026-08-07 22:48:26 on dist-test-slave-csg5"
server_key: "c6bdc1ab42947efd4dabf10b00029474"
server_key_iv: "8a8e6ea6a8b27c2e5ed2c0e291678f94"
server_key_version: "encryptionkey@0"
I20260807 22:48:26.437644 7404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:26.455647 7404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:26.456693 7404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:26.457047 7404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:26.480517 7404 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:35097
I20260807 22:48:26.480525 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:35097 every 8 connection(s)
I20260807 22:48:26.482196 7404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:26.486572 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:26.488521 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7404
I20260807 22:48:26.488729 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:26.489315 20522 external_mini_cluster.cc:1428] Setting key ec97eb8168be54d76781db212a28be5e
I20260807 22:48:26.501921 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: Bootstrap starting.
I20260807 22:48:26.506459 7477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:26.507417 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:26.491737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43130 (local address 127.20.10.190:35097)
0807 22:48:26.493028 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:26.493044 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:26.493103 (+ 59us) server_negotiation.cc:407] Connection header received
0807 22:48:26.495643 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:26.495677 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:26.496323 (+ 646us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:26.496784 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:26.497371 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:26.499555 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:26.501464 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:26.502578 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:26.502811 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:26.504194 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:26.504207 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:26.504229 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:26.505018 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:26.505169 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:26.505198 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:26.505325 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:26.505567 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 22:48:26.505923 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":460,"threads_started":1}
I20260807 22:48:26.507920 7477 log.cc:824] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:26.511082 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: No bootstrap required, opened a new log
I20260807 22:48:26.519343 7477 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } }
I20260807 22:48:26.519848 7477 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:26.519972 7477 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6f85eb7b04ea2ab664ea7feb7cf56, State: Initialized, Role: FOLLOWER
I20260807 22:48:26.520581 7477 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } }
I20260807 22:48:26.520862 7477 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:26.521050 7477 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:26.521260 7477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:26.523814 7477 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } }
I20260807 22:48:26.524362 7477 leader_election.cc:302] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4f6f85eb7b04ea2ab664ea7feb7cf56; no voters:
I20260807 22:48:26.525000 7477 leader_election.cc:288] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:26.525190 7482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:26.526039 7482 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6f85eb7b04ea2ab664ea7feb7cf56, State: Running, Role: LEADER
I20260807 22:48:26.526825 7482 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } }
I20260807 22:48:26.527436 7477 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:26.529809 7484 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4f6f85eb7b04ea2ab664ea7feb7cf56. Latest consensus state: current_term: 1 leader_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } } }
I20260807 22:48:26.530309 7484 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:26.530053 7483 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f6f85eb7b04ea2ab664ea7feb7cf56" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 35097 } } }
I20260807 22:48:26.530531 7483 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:26.531792 7492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:26.538805 7492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:26.550355 7492 catalog_manager.cc:1347] Generated new cluster ID: 693717d6675b4cdfbeb0278dab5f9cef
I20260807 22:48:26.550509 7492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:26.563406 7492 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:26.565521 7492 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:26.573285 7492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: Generated new TSK 0
I20260807 22:48:26.574481 7492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:26.584237 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--builtin_ntp_servers=127.20.10.148:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:26.997953 7503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:26.998286 7503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:26.998466 7503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:27.007076 7503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:27.007197 7503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:27.007373 7503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:27.017405 7503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:27.018980 7503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:27.020929 7503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:27.036717 7512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:27.036932 7509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:27.036953 7508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:27.038038 7503 server_base.cc:1034] running on GCE node
I20260807 22:48:27.039002 7503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:27.040428 7503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:27.041687 7503 hybrid_clock.cc:648] HybridClock initialized: now 1786142907041628 us; error 55 us; skew 500 ppm
I20260807 22:48:27.042145 7503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:27.045037 7503 webserver.cc:460] Webserver started at http://127.20.10.129:38877/ using document root <none> and password file <none>
I20260807 22:48:27.045912 7503 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:27.046053 7503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:27.046509 7503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:27.050381 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "50fdc9d480d6400a972bb106b8ca3af1"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "e67a81e55da37eef09a10994148c867c"
server_key_iv: "b95d61b9cd27d0f36568d5ee811011d4"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.051623 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "50fdc9d480d6400a972bb106b8ca3af1"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "e67a81e55da37eef09a10994148c867c"
server_key_iv: "b95d61b9cd27d0f36568d5ee811011d4"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.059811 7503 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260807 22:48:27.064128 7520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.065965 7503 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:48:27.066146 7503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "50fdc9d480d6400a972bb106b8ca3af1"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "e67a81e55da37eef09a10994148c867c"
server_key_iv: "b95d61b9cd27d0f36568d5ee811011d4"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.066362 7503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:27.083515 7503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:27.084515 7503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:27.084861 7503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:27.086033 7503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:27.087904 7503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:27.088011 7503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.088109 7503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:27.088176 7503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.131155 7503 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:41173
I20260807 22:48:27.131228 7643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:41173 every 8 connection(s)
I20260807 22:48:27.132882 7503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:27.140852 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7503
I20260807 22:48:27.141088 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:27.141762 20522 external_mini_cluster.cc:1428] Setting key cc50abcf778954c5238b23be3ea6ac56
I20260807 22:48:27.145426 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--builtin_ntp_servers=127.20.10.148:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:27.150883 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:27.136410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:45657 (local address 127.20.10.190:35097)
0807 22:48:27.136962 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:27.136970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:27.138467 (+ 1497us) server_negotiation.cc:407] Connection header received
0807 22:48:27.141074 (+ 2607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:27.141081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:27.141218 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:27.141361 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:27.143047 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.143912 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.147105 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.147617 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.147732 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:27.149805 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:27.149811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:27.149815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:27.149849 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:27.149895 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:27.149908 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:27.149989 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:27.150387 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 22:48:27.150494 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":272,"threads_started":1}
I20260807 22:48:27.152571 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:27.137092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35097 (local address 127.20.10.129:45657)
0807 22:48:27.138270 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:27.138317 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:27.140639 (+ 2322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:27.141710 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:27.141723 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:27.141873 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:27.142820 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:27.142850 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.144298 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.144304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:27.146854 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:27.146880 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.147925 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.147935 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:27.148382 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:27.149303 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:27.149310 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:27.149381 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:27.149515 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:27.150023 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:27.150039 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:27.150052 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:48:27.150295 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 22:48:27.150851 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":352,"threads_started":1}
I20260807 22:48:27.154701 7644 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35097
I20260807 22:48:27.155105 7644 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:27.156282 7644 heartbeater.cc:511] Master 127.20.10.190:35097 requested a full tablet report, sending...
I20260807 22:48:27.159173 7440 ts_manager.cc:194] Registered new tserver with Master: 50fdc9d480d6400a972bb106b8ca3af1 (127.20.10.129:41173)
I20260807 22:48:27.161636 7440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.129:45657
W20260807 22:48:27.529410 7648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:27.529728 7648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:27.529826 7648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:27.537868 7648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:27.537992 7648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:27.538105 7648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:27.548405 7648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:27.549867 7648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:27.551877 7648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:27.567823 7654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:27.567823 7658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:27.567845 7656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:27.569443 7648 server_base.cc:1034] running on GCE node
I20260807 22:48:27.570242 7648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:27.571894 7648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:27.573201 7648 hybrid_clock.cc:648] HybridClock initialized: now 1786142907573137 us; error 67 us; skew 500 ppm
I20260807 22:48:27.573693 7648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:27.576748 7648 webserver.cc:460] Webserver started at http://127.20.10.130:44313/ using document root <none> and password file <none>
I20260807 22:48:27.577629 7648 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:27.577790 7648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:27.578156 7648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:27.581524 7648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "0fa11f3f58ab5071bc8c305c026e7f93"
server_key_iv: "f61723dbe7b0d740ab0e8194caa79e05"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.582811 7648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "0fa11f3f58ab5071bc8c305c026e7f93"
server_key_iv: "f61723dbe7b0d740ab0e8194caa79e05"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.589002 7648 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:48:27.593240 7665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.594988 7648 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:48:27.595235 7648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3"
format_stamp: "Formatted at 2026-08-07 22:48:27 on dist-test-slave-csg5"
server_key: "0fa11f3f58ab5071bc8c305c026e7f93"
server_key_iv: "f61723dbe7b0d740ab0e8194caa79e05"
server_key_version: "encryptionkey@0"
I20260807 22:48:27.595510 7648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:27.612732 7648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:27.613757 7648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:27.614138 7648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:27.615402 7648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:27.617511 7648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:27.617684 7648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.617821 7648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:27.617921 7648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:27.661494 7648 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:38591
I20260807 22:48:27.661607 7785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:38591 every 8 connection(s)
I20260807 22:48:27.663197 7648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:27.666510 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7648
I20260807 22:48:27.666747 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:27.667322 20522 external_mini_cluster.cc:1428] Setting key 258b351572817a5b96a61a76284455b9
I20260807 22:48:27.670537 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--builtin_ntp_servers=127.20.10.148:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:27.681269 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:27.667000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:38809 (local address 127.20.10.190:35097)
0807 22:48:27.667546 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:27.667553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:27.668746 (+ 1193us) server_negotiation.cc:407] Connection header received
0807 22:48:27.671201 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:27.671210 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:27.671326 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:27.671469 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:27.673344 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.674757 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.677071 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.677909 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.678104 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:27.679926 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:27.679936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:27.679944 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:27.679978 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:27.680049 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:27.680071 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:27.680245 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:27.680590 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 22:48:27.680784 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":252,"threads_started":1}
I20260807 22:48:27.682296 7789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:27.667135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35097 (local address 127.20.10.130:38809)
0807 22:48:27.668543 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:27.668597 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:27.670780 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:27.672018 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:27.672028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:27.672155 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:27.673075 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:27.673099 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.674991 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.674996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:27.676622 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:27.676874 (+ 252us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:27.678302 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:27.678310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:27.678571 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:27.679504 (+ 933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:27.679507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:27.679555 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:27.679664 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:27.680292 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:27.680305 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:27.680313 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:48:27.680526 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 22:48:27.680923 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":500,"threads_started":1}
I20260807 22:48:27.684672 7786 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35097
I20260807 22:48:27.685179 7786 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:27.686295 7786 heartbeater.cc:511] Master 127.20.10.190:35097 requested a full tablet report, sending...
I20260807 22:48:27.688211 7440 ts_manager.cc:194] Registered new tserver with Master: 2e1b6ef7ac7f4f21ae5a9539f04d33a3 (127.20.10.130:38591)
I20260807 22:48:27.689474 7440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.130:38809
W20260807 22:48:28.053495 7790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:28.053817 7790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:28.053975 7790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:28.061790 7790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:28.061906 7790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:28.062090 7790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:28.072228 7790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:35097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:28.073696 7790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:28.075765 7790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:28.091002 7795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:28.090718 7796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:28.090953 7798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:28.092861 7790 server_base.cc:1034] running on GCE node
I20260807 22:48:28.093858 7790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:28.095116 7790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:28.096423 7790 hybrid_clock.cc:648] HybridClock initialized: now 1786142908096380 us; error 51 us; skew 500 ppm
I20260807 22:48:28.096879 7790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:28.099788 7790 webserver.cc:460] Webserver started at http://127.20.10.131:36669/ using document root <none> and password file <none>
I20260807 22:48:28.100670 7790 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:28.100827 7790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:28.101261 7790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:28.104676 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "6504a4f422824cbf8ecc2de65f6ee8a5"
format_stamp: "Formatted at 2026-08-07 22:48:28 on dist-test-slave-csg5"
server_key: "388b532657e1d0e58848cef77571c9e7"
server_key_iv: "93bdad782cdee76067b9c27f5b341d03"
server_key_version: "encryptionkey@0"
I20260807 22:48:28.105614 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "6504a4f422824cbf8ecc2de65f6ee8a5"
format_stamp: "Formatted at 2026-08-07 22:48:28 on dist-test-slave-csg5"
server_key: "388b532657e1d0e58848cef77571c9e7"
server_key_iv: "93bdad782cdee76067b9c27f5b341d03"
server_key_version: "encryptionkey@0"
I20260807 22:48:28.111897 7790 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 22:48:28.116760 7806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:28.118465 7790 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:48:28.118747 7790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "6504a4f422824cbf8ecc2de65f6ee8a5"
format_stamp: "Formatted at 2026-08-07 22:48:28 on dist-test-slave-csg5"
server_key: "388b532657e1d0e58848cef77571c9e7"
server_key_iv: "93bdad782cdee76067b9c27f5b341d03"
server_key_version: "encryptionkey@0"
I20260807 22:48:28.119029 7790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:28.140733 7790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:28.141682 7790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:28.142094 7790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:28.143263 7790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:28.144927 7790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:28.145053 7790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:28.145200 7790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:28.145290 7790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:28.165787 7644 heartbeater.cc:503] Master 127.20.10.190:35097 was elected leader, sending a full tablet report...
I20260807 22:48:28.184278 7790 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:35527
I20260807 22:48:28.184413 7928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:35527 every 8 connection(s)
I20260807 22:48:28.185786 7790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:28.193346 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 7790
I20260807 22:48:28.193580 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:28.194186 20522 external_mini_cluster.cc:1428] Setting key 12a1790c7dcbfacfa262e4dd5f5be3cd
I20260807 22:48:28.201290 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.188762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:53061 (local address 127.20.10.190:35097)
0807 22:48:28.189209 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.189214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.190760 (+ 1546us) server_negotiation.cc:407] Connection header received
0807 22:48:28.192743 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.192748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.192828 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.192936 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:28.194839 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.196036 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.197966 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.198391 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.198497 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.200202 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:28.200208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:28.200211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:28.200235 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:28.200289 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:28.200333 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:28.200499 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.200869 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.200968 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":222,"threads_started":1}
I20260807 22:48:28.202453 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.189432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:35097 (local address 127.20.10.131:53061)
0807 22:48:28.190508 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.190554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.192386 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.193386 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.193401 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.193555 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:28.194506 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.194537 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.196422 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.196426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.197800 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.197816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.198636 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.198642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.198968 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.199779 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:28.199782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:28.199831 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:28.199933 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:28.200598 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:28.200619 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:28.200626 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.200800 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.201197 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":293,"threads_started":1}
I20260807 22:48:28.204304 7929 heartbeater.cc:347] Connected to a master server at 127.20.10.190:35097
I20260807 22:48:28.204682 7929 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:28.205595 7929 heartbeater.cc:511] Master 127.20.10.190:35097 requested a full tablet report, sending...
I20260807 22:48:28.207337 7440 ts_manager.cc:194] Registered new tserver with Master: 6504a4f422824cbf8ecc2de65f6ee8a5 (127.20.10.131:35527)
I20260807 22:48:28.208178 7440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.131:53061
I20260807 22:48:28.210830 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:28.223078 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.215826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.20.10.190:35097)
0807 22:48:28.216116 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.216123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.216512 (+ 389us) server_negotiation.cc:407] Connection header received
0807 22:48:28.216652 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.216656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.216735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.216848 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:28.217535 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.218294 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.220002 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.220540 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.220666 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.222087 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:28.222092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:28.222096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:28.222121 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:28.222170 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:28.222184 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:28.222290 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.222552 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.222808 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 22:48:28.228725 7440 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:28.237253 7440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:28.279253 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.253112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.20.10.129:41173)
0807 22:48:28.266039 (+ 12927us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.266053 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.266097 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:48:28.266255 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.266268 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.266647 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.266953 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:28.269364 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.271599 (+ 2235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.276050 (+ 4451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.277757 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.277950 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.278301 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.278589 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.278838 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12651,"thread_start_us":12467,"threads_started":1}
I20260807 22:48:28.279256 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.252472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50586)
0807 22:48:28.264736 (+ 12264us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.264823 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.265147 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.268307 (+ 3160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.268318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.268404 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:28.269043 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.269065 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.271815 (+ 2750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.271823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.275731 (+ 3908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.275745 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.278018 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.278025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.278278 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.278348 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.278538 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.278819 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11827,"spinlock_wait_cycles":22400,"thread_start_us":18065,"threads_started":1}
I20260807 22:48:28.280725 7949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.252472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38591 (local address 127.0.0.1:35960)
0807 22:48:28.268154 (+ 15682us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.268186 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.268397 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.272060 (+ 3663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.272064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.272101 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:28.273006 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.273024 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.274633 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.274640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.276997 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.277015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.279757 (+ 2742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.279764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.279973 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.280011 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.280103 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.280283 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15456,"mutex_wait_us":258,"spinlock_wait_cycles":6912,"thread_start_us":14531,"threads_started":1}
I20260807 22:48:28.281903 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.252548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.20.10.130:38591)
0807 22:48:28.271284 (+ 18736us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.271294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.271321 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:48:28.271438 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.271449 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.271771 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.272729 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:28.273314 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.274469 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.277319 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.279068 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.279233 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.281231 (+ 1998us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.281323 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.281532 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18541,"thread_start_us":18430,"threads_started":1}
I20260807 22:48:28.282447 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.255902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35527 (local address 127.0.0.1:32908)
0807 22:48:28.270908 (+ 15006us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.270934 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.271325 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.272220 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.272225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.272260 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:28.273286 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.273303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.275888 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.275895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.278578 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.278597 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.281415 (+ 2818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.281423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.281616 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.281769 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.281892 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.282064 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14849,"mutex_wait_us":40,"thread_start_us":12616,"threads_started":1}
I20260807 22:48:28.282997 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.256153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.20.10.131:35527)
0807 22:48:28.270267 (+ 14114us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.270281 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.271249 (+ 968us) server_negotiation.cc:407] Connection header received
0807 22:48:28.271618 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.271625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.271834 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.272028 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:28.273518 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.275434 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.278897 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.280796 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.280964 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.281260 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.282173 (+ 913us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.282374 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13770,"thread_start_us":13555,"threads_started":1}
I20260807 22:48:28.283913 7574 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c (DEFAULT_TABLE table=test-table [id=8fc9f0adce734c8a81ed2b20aec992f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:28.285847 7574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:28.286339 7717 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c (DEFAULT_TABLE table=test-table [id=8fc9f0adce734c8a81ed2b20aec992f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:28.287739 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:28.287204 7859 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c (DEFAULT_TABLE table=test-table [id=8fc9f0adce734c8a81ed2b20aec992f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:28.288807 7859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:28.301541 7951 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Bootstrap starting.
I20260807 22:48:28.303929 7952 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Bootstrap starting.
I20260807 22:48:28.306423 7954 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Bootstrap starting.
I20260807 22:48:28.308198 7951 tablet_bootstrap.cc:654] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:28.308202 7952 tablet_bootstrap.cc:654] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:28.309821 7951 log.cc:824] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:28.309821 7952 log.cc:824] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:28.310242 7954 tablet_bootstrap.cc:654] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:28.311842 7954 log.cc:824] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:28.314371 7952 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: No bootstrap required, opened a new log
I20260807 22:48:28.314819 7951 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: No bootstrap required, opened a new log
I20260807 22:48:28.314805 7952 ts_tablet_manager.cc:1397] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.004s
I20260807 22:48:28.315140 7951 ts_tablet_manager.cc:1397] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260807 22:48:28.317534 7954 tablet_bootstrap.cc:492] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: No bootstrap required, opened a new log
I20260807 22:48:28.317934 7954 ts_tablet_manager.cc:1397] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260807 22:48:28.324736 7952 raft_consensus.cc:348] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.325428 7952 raft_consensus.cc:374] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:28.325575 7952 raft_consensus.cc:729] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50fdc9d480d6400a972bb106b8ca3af1, State: Initialized, Role: FOLLOWER
I20260807 22:48:28.326340 7952 consensus_queue.cc:260] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.326575 7951 raft_consensus.cc:348] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.327085 7951 raft_consensus.cc:374] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:28.327185 7951 raft_consensus.cc:729] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1b6ef7ac7f4f21ae5a9539f04d33a3, State: Initialized, Role: FOLLOWER
I20260807 22:48:28.327896 7951 consensus_queue.cc:260] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.328881 7952 ts_tablet_manager.cc:1428] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 22:48:28.330193 7786 heartbeater.cc:503] Master 127.20.10.190:35097 was elected leader, sending a full tablet report...
I20260807 22:48:28.330116 7954 raft_consensus.cc:348] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.330593 7954 raft_consensus.cc:374] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:28.330739 7954 raft_consensus.cc:729] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6504a4f422824cbf8ecc2de65f6ee8a5, State: Initialized, Role: FOLLOWER
I20260807 22:48:28.330968 7951 ts_tablet_manager.cc:1428] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260807 22:48:28.331462 7954 consensus_queue.cc:260] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.333345 7929 heartbeater.cc:503] Master 127.20.10.190:35097 was elected leader, sending a full tablet report...
I20260807 22:48:28.333539 7958 raft_consensus.cc:482] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:28.334007 7954 ts_tablet_manager.cc:1428] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20260807 22:48:28.333863 7958 raft_consensus.cc:504] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.336258 7958 leader_election.cc:288] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e1b6ef7ac7f4f21ae5a9539f04d33a3 (127.20.10.130:38591), 6504a4f422824cbf8ecc2de65f6ee8a5 (127.20.10.131:35527)
I20260807 22:48:28.351982 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.336386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38591 (local address 127.20.10.129:36005)
0807 22:48:28.343404 (+ 7018us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.343431 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.343687 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.344178 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.344183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.344220 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:28.344673 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.344684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.346274 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.346278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.349429 (+ 3151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.349487 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.350975 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.350982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.351193 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.351239 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.351368 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.351530 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6831,"mutex_wait_us":144,"thread_start_us":7542,"threads_started":1}
I20260807 22:48:28.352090 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.336889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:36005 (local address 127.20.10.130:38591)
0807 22:48:28.337118 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.337128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.343593 (+ 6465us) server_negotiation.cc:407] Connection header received
0807 22:48:28.343869 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.343879 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.344003 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.344127 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:28.344840 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.346021 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.349803 (+ 3782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.350722 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.350969 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.351060 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.351428 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.351653 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:48:28.353482 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.337385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:47845 (local address 127.20.10.131:35527)
0807 22:48:28.337716 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.337726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.344338 (+ 6612us) server_negotiation.cc:407] Connection header received
0807 22:48:28.344584 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.344590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.344676 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.344809 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:28.345367 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.346426 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.349844 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.352028 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.352277 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.352387 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.353052 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.353188 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:48:28.353724 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.336770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35527 (local address 127.20.10.129:47845)
0807 22:48:28.344224 (+ 7454us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.344244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:28.344407 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:28.344824 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:28.344829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:28.344857 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:28.345203 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.345212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.346562 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.346569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.349470 (+ 2901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:28.349487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.352548 (+ 3061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.352557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:28.352806 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.352858 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 22:48:28.352979 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:48:28.353168 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7287,"mutex_wait_us":97,"thread_start_us":6212,"threads_started":1}
I20260807 22:48:28.355062 7881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ecfbc7d846b282fcf99c8b2eb53c" candidate_uuid: "50fdc9d480d6400a972bb106b8ca3af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" is_pre_election: true
I20260807 22:48:28.355763 7881 raft_consensus.cc:2454] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fdc9d480d6400a972bb106b8ca3af1 in term 0.
I20260807 22:48:28.356662 7522 leader_election.cc:302] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fdc9d480d6400a972bb106b8ca3af1, 6504a4f422824cbf8ecc2de65f6ee8a5; no voters:
I20260807 22:48:28.357299 7958 raft_consensus.cc:2781] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:28.357491 7958 raft_consensus.cc:482] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:28.357611 7958 raft_consensus.cc:3037] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:28.357499 7739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ecfbc7d846b282fcf99c8b2eb53c" candidate_uuid: "50fdc9d480d6400a972bb106b8ca3af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" is_pre_election: true
I20260807 22:48:28.358172 7739 raft_consensus.cc:2454] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fdc9d480d6400a972bb106b8ca3af1 in term 0.
I20260807 22:48:28.360925 7958 raft_consensus.cc:504] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.361740 7958 leader_election.cc:288] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [CANDIDATE]: Term 1 election: Requested vote from peers 2e1b6ef7ac7f4f21ae5a9539f04d33a3 (127.20.10.130:38591), 6504a4f422824cbf8ecc2de65f6ee8a5 (127.20.10.131:35527)
I20260807 22:48:28.362370 7739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ecfbc7d846b282fcf99c8b2eb53c" candidate_uuid: "50fdc9d480d6400a972bb106b8ca3af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3"
I20260807 22:48:28.362473 7881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ecfbc7d846b282fcf99c8b2eb53c" candidate_uuid: "50fdc9d480d6400a972bb106b8ca3af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5"
I20260807 22:48:28.362608 7739 raft_consensus.cc:3037] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:28.362824 7881 raft_consensus.cc:3037] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:28.365243 7881 raft_consensus.cc:2454] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fdc9d480d6400a972bb106b8ca3af1 in term 1.
I20260807 22:48:28.365271 7739 raft_consensus.cc:2454] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fdc9d480d6400a972bb106b8ca3af1 in term 1.
I20260807 22:48:28.365733 7522 leader_election.cc:302] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fdc9d480d6400a972bb106b8ca3af1, 6504a4f422824cbf8ecc2de65f6ee8a5; no voters:
I20260807 22:48:28.366062 7958 raft_consensus.cc:2781] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:28.366523 7958 raft_consensus.cc:686] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 1 LEADER]: Becoming Leader. State: Replica: 50fdc9d480d6400a972bb106b8ca3af1, State: Running, Role: LEADER
I20260807 22:48:28.367187 7958 consensus_queue.cc:237] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } }
I20260807 22:48:28.372427 7439 catalog_manager.cc:5495] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50fdc9d480d6400a972bb106b8ca3af1 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "50fdc9d480d6400a972bb106b8ca3af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50fdc9d480d6400a972bb106b8ca3af1" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 } health_report { overall_health: UNKNOWN } } }
W20260807 22:48:28.385881 7645 tablet.cc:2369] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:28.415910 7787 tablet.cc:2369] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:28.417961 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.411010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.20.10.129:41173)
0807 22:48:28.411374 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.411379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.411396 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:48:28.411497 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.411501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.411613 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.411809 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:28.412219 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.412969 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.414595 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.415292 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.415449 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.416431 (+ 982us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:28.417205 (+ 774us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:28.417364 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.417581 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.417684 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:48:28.428589 7881 raft_consensus.cc:1261] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 1 FOLLOWER]: Refusing update from remote peer 50fdc9d480d6400a972bb106b8ca3af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:28.428589 7739 raft_consensus.cc:1261] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 1 FOLLOWER]: Refusing update from remote peer 50fdc9d480d6400a972bb106b8ca3af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:28.430032 7963 consensus_queue.cc:1035] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1b6ef7ac7f4f21ae5a9539f04d33a3" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:28.430588 7958 consensus_queue.cc:1035] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6504a4f422824cbf8ecc2de65f6ee8a5" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:48:28.437805 7930 tablet.cc:2369] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:28.467322 7440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:28.467669 7440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:28.472651 7440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4f6f85eb7b04ea2ab664ea7feb7cf56: Sending DeleteTablet for 3 replicas of tablet a5e3ecfbc7d846b282fcf99c8b2eb53c
I20260807 22:48:28.474085 7717 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:28 UTC) from {username='slave'} at 127.0.0.1:35960
I20260807 22:48:28.474767 7859 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:28 UTC) from {username='slave'} at 127.0.0.1:32908
I20260807 22:48:28.476608 7980 tablet_replica.cc:330] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: stopping tablet replica
I20260807 22:48:28.476665 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7503
I20260807 22:48:28.476311 7574 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:28 UTC) from {username='slave'} at 127.0.0.1:50586
I20260807 22:48:28.477195 7980 raft_consensus.cc:2229] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:28.477911 7980 raft_consensus.cc:2258] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:28.480881 7980 ts_tablet_manager.cc:1905] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:28.489494 7980 ts_tablet_manager.cc:1918] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:28.489755 7980 log.cc:1196] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/a5e3ecfbc7d846b282fcf99c8b2eb53c
I20260807 22:48:28.490423 7980 ts_tablet_manager.cc:1939] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 6504a4f422824cbf8ecc2de65f6ee8a5: Deleting consensus metadata
I20260807 22:48:28.491777 7981 tablet_replica.cc:330] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: stopping tablet replica
I20260807 22:48:28.492780 7421 catalog_manager.cc:4841] TS 6504a4f422824cbf8ecc2de65f6ee8a5 (127.20.10.131:35527): tablet a5e3ecfbc7d846b282fcf99c8b2eb53c (table test-table [id=8fc9f0adce734c8a81ed2b20aec992f5]) successfully deleted
I20260807 22:48:28.492790 7981 raft_consensus.cc:2229] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:28.495270 7981 raft_consensus.cc:2258] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:28.499387 7979 tablet_replica.cc:330] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: stopping tablet replica
I20260807 22:48:28.499977 7979 raft_consensus.cc:2229] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:28.500509 7981 ts_tablet_manager.cc:1905] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 50fdc9d480d6400a972bb106b8ca3af1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:28.500727 7979 raft_consensus.cc:2258] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:28.503427 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.479063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50608 (local address 127.20.10.129:41173)
0807 22:48:28.479308 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.479318 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.491331 (+ 12013us) server_negotiation.cc:407] Connection header received
0807 22:48:28.493366 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.493374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.493521 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.493644 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:28.494619 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.496172 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.499114 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.499752 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.499924 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.501485 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:28.501497 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:28.501520 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:28.502361 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:28.502421 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:28.502447 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:28.502552 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.502861 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.502987 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:48:28.504964 7638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:28.508217 7979 ts_tablet_manager.cc:1905] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:28.513931 7979 ts_tablet_manager.cc:1918] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:28.514101 7979 log.cc:1196] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/a5e3ecfbc7d846b282fcf99c8b2eb53c
I20260807 22:48:28.514595 7979 ts_tablet_manager.cc:1939] T a5e3ecfbc7d846b282fcf99c8b2eb53c P 2e1b6ef7ac7f4f21ae5a9539f04d33a3: Deleting consensus metadata
I20260807 22:48:28.516513 7423 catalog_manager.cc:4841] TS 2e1b6ef7ac7f4f21ae5a9539f04d33a3 (127.20.10.130:38591): tablet a5e3ecfbc7d846b282fcf99c8b2eb53c (table test-table [id=8fc9f0adce734c8a81ed2b20aec992f5]) successfully deleted
I20260807 22:48:28.609691 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7503
I20260807 22:48:28.634603 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7648
W20260807 22:48:28.634624 7420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.10.129:41173 (error 108)
I20260807 22:48:28.637152 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.636697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50614)
0807 22:48:28.636879 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.636999 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:48:28.637398 7420 catalog_manager.cc:4584] TS 50fdc9d480d6400a972bb106b8ca3af1 (127.20.10.129:41173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5e3ecfbc7d846b282fcf99c8b2eb53c: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
I20260807 22:48:28.642503 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.635575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.20.10.130:38591)
0807 22:48:28.635767 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.635773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.635846 (+ 73us) server_negotiation.cc:407] Connection header received
0807 22:48:28.636074 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.636081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.636187 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.636301 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:28.636741 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.637677 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.639237 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.639677 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.639834 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.641024 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:28.641034 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:28.641049 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:28.641650 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:28.641699 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:28.641718 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:28.641867 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.641978 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.642096 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 22:48:28.643501 7781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:28.691555 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.690922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50628)
0807 22:48:28.691177 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.691298 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 22:48:28.692852 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.692390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50630)
0807 22:48:28.692572 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.692667 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 22:48:28.740959 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7648
I20260807 22:48:28.753928 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.753395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50640)
0807 22:48:28.753638 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.753747 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 22:48:28.755091 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.754721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50646)
0807 22:48:28.754876 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.754959 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 22:48:28.764696 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7790
I20260807 22:48:28.772837 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.765711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.20.10.131:35527)
0807 22:48:28.765911 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:28.765916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:28.765936 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:48:28.766263 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:28.766273 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:28.766373 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:28.766483 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:28.767071 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.767886 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.769497 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:28.769909 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:28.770007 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:28.771372 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:28.771381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:28.771396 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:28.772000 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:28.772050 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:28.772072 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:28.772168 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:28.772470 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 22:48:28.772576 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:48:28.773550 7923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:28.836104 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.835572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50660)
0807 22:48:28.835834 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.835935 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:48:28.837466 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:28.836976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:41173 (local address 127.0.0.1:50674)
0807 22:48:28.837255 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:28.837338 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.129:41173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 22:48:28.869378 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7790
I20260807 22:48:28.891060 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 7404
I20260807 22:48:28.892549 7472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:29.011570 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 7404
2026-08-07T22:48:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3143 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:29 dist-test-slave-csg5 krb5kdc[7991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:29 dist-test-slave-csg5 krb5kdc[7991](info): set up 2 sockets
Aug 07 22:48:29 dist-test-slave-csg5 krb5kdc[7991](info): commencing operation
krb5kdc: starting...
W20260807 22:48:30.332746 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:30 dist-test-slave-csg5 krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:48:30.715018 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46515
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:31.092175 8004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:31.092597 8004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:31.092777 8004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:31.100660 8004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:31.100847 8004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:31.100968 8004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:31.101080 8004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:31.101172 8004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:31.112095 8004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:31.113713 8004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:31.115338 8004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:31.130118 8011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:31.130347 8013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:31.130672 8010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:31.132107 8004 server_base.cc:1034] running on GCE node
I20260807 22:48:31.133697 8004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:31.135830 8004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:31.137472 8004 hybrid_clock.cc:648] HybridClock initialized: now 1786142911137406 us; error 91 us; skew 500 ppm
I20260807 22:48:31.138019 8004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:31 dist-test-slave-csg5 krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:31.147956 8004 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:48:31.150202 8004 webserver.cc:460] Webserver started at http://127.20.10.190:36855/ using document root <none> and password file <none>
I20260807 22:48:31.151162 8004 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:31.151361 8004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:31.151715 8004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:31.155702 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "b1864ca27be44eebac67e1989b337810"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0a4ae0ec2dd21857e767c2d2fcd4a3b5"
server_key_iv: "5b03c4cd36c58309fa436117d3b5b09b"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.156605 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "b1864ca27be44eebac67e1989b337810"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0a4ae0ec2dd21857e767c2d2fcd4a3b5"
server_key_iv: "5b03c4cd36c58309fa436117d3b5b09b"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.163095 8004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 22:48:31.167805 8020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:31.169771 8004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:48:31.170042 8004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "b1864ca27be44eebac67e1989b337810"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0a4ae0ec2dd21857e767c2d2fcd4a3b5"
server_key_iv: "5b03c4cd36c58309fa436117d3b5b09b"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.170307 8004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:31.189502 8004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:31.196069 8004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:31.196450 8004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:31.221244 8004 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46515
I20260807 22:48:31.221314 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46515 every 8 connection(s)
I20260807 22:48:31.222942 8004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:31.223658 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8004
I20260807 22:48:31.223938 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:31.224915 20522 external_mini_cluster.cc:1428] Setting key 2060cac607f8327dcd4de8f8d6fe899f
I20260807 22:48:31.228618 8076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:48:31 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:31.244359 8076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: Bootstrap starting.
I20260807 22:48:31.249200 8076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:31.250581 8076 log.cc:824] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:31.255229 8076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: No bootstrap required, opened a new log
I20260807 22:48:31.255889 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:31.227442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.20.10.190:46515)
0807 22:48:31.228493 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:31.228506 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:31.228555 (+ 49us) server_negotiation.cc:407] Connection header received
0807 22:48:31.230482 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:31.230508 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:31.231026 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:31.231408 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:31.233683 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:31.235648 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:31.237800 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:31.238630 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:31.238885 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:31.244837 (+ 5952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:31.244851 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:31.244876 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:31.244938 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:31.249587 (+ 4649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:31.250549 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:31.250560 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:31.250576 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:31.250862 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:31.251787 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:31.251792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:31.251795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:31.252656 (+ 861us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:31.253004 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:31.253504 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 22:48:31.254068 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":462,"threads_started":1}
I20260807 22:48:31.264808 8076 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } }
I20260807 22:48:31.265316 8076 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:31.265395 8076 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1864ca27be44eebac67e1989b337810, State: Initialized, Role: FOLLOWER
I20260807 22:48:31.265955 8076 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } }
I20260807 22:48:31.266166 8076 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:31.266273 8076 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:31.266410 8076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:31.268829 8076 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } }
I20260807 22:48:31.269405 8076 leader_election.cc:302] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1864ca27be44eebac67e1989b337810; no voters:
I20260807 22:48:31.270061 8076 leader_election.cc:288] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:31.270345 8083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:31.271982 8083 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [term 1 LEADER]: Becoming Leader. State: Replica: b1864ca27be44eebac67e1989b337810, State: Running, Role: LEADER
I20260807 22:48:31.272619 8076 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:31.272624 8083 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } }
I20260807 22:48:31.276343 8084 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1864ca27be44eebac67e1989b337810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } } }
I20260807 22:48:31.276706 8085 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1864ca27be44eebac67e1989b337810. Latest consensus state: current_term: 1 leader_uuid: "b1864ca27be44eebac67e1989b337810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1864ca27be44eebac67e1989b337810" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46515 } } }
I20260807 22:48:31.277046 8085 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:31.277052 8084 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:31.279899 8091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:31.286484 8091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:31.297894 8091 catalog_manager.cc:1347] Generated new cluster ID: 302bf965b6804ff095e7b60ad554cd68
I20260807 22:48:31.298054 8091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:31.310827 8091 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:31.312451 8091 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:31.323251 8091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: Generated new TSK 0
I20260807 22:48:31.324254 8091 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:48:31.429664 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:31.850915 8107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:31.851317 8107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:31.851544 8107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:31.859814 8107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:31.859943 8107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:31.860061 8107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:31.870818 8107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:31.872218 8107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:31.873899 8107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:31.890556 8115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:31.890720 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:31.890694 8113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:31.891651 8107 server_base.cc:1034] running on GCE node
I20260807 22:48:31.892750 8107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:31.893947 8107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:31.895386 8107 hybrid_clock.cc:648] HybridClock initialized: now 1786142911895395 us; error 115 us; skew 500 ppm
I20260807 22:48:31.895823 8107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:31 dist-test-slave-csg5 krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:31.900337 8107 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:48:31.902717 8107 webserver.cc:460] Webserver started at http://127.20.10.129:42383/ using document root <none> and password file <none>
I20260807 22:48:31.903568 8107 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:31.903759 8107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:31.904145 8107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:31.907660 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "476901e77abf4b3fa3f2ca3186a191d9"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0bb1e0ccf75c9561b53e78eec44c50e1"
server_key_iv: "0b616416be788dd139194466e9b348f7"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.908601 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "476901e77abf4b3fa3f2ca3186a191d9"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0bb1e0ccf75c9561b53e78eec44c50e1"
server_key_iv: "0b616416be788dd139194466e9b348f7"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.915555 8107 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:48:31.920783 8123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:31.922886 8107 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260807 22:48:31.923113 8107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "476901e77abf4b3fa3f2ca3186a191d9"
format_stamp: "Formatted at 2026-08-07 22:48:31 on dist-test-slave-csg5"
server_key: "0bb1e0ccf75c9561b53e78eec44c50e1"
server_key_iv: "0b616416be788dd139194466e9b348f7"
server_key_version: "encryptionkey@0"
I20260807 22:48:31.923350 8107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:31.959728 8107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:31.966138 8107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:31.966463 8107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:31.967787 8107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:31.969478 8107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:31.969576 8107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:31.969691 8107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:31.969750 8107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:32.012470 8107 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:33099
I20260807 22:48:32.012539 8248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:33099 every 8 connection(s)
I20260807 22:48:32.014063 8107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:32.023277 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8107
I20260807 22:48:32.023488 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:32.024060 20522 external_mini_cluster.cc:1428] Setting key 219bcae6dd76bf4b9f1452c4ee667acb
Aug 07 22:48:32 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:32.041406 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:32.017638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:33061 (local address 127.20.10.190:46515)
0807 22:48:32.018289 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:32.018295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:32.019490 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 22:48:32.021966 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:32.021972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:32.022107 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:32.022263 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:32.024546 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.025827 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.028326 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.028942 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.029102 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:32.034001 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:32.034007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:32.034011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:32.034055 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:32.037301 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:32.038369 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:32.038381 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:32.038384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:32.038548 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:32.039178 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:32.039183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:32.039186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:32.039540 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:32.039740 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:32.040633 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 22:48:32.040970 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":349,"threads_started":1}
I20260807 22:48:32.042631 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:32.018189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46515 (local address 127.20.10.129:33061)
0807 22:48:32.019268 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:32.019330 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:32.021557 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:32.022597 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:32.022607 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:32.023386 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:32.024238 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:32.024262 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.026246 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.026251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:32.028111 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:32.028132 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.029250 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.029260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:32.029528 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:32.030353 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:32.030385 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:32.033627 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:32.037554 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:32.037567 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:32.037598 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:32.038175 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:32.038732 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:32.038738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:32.038743 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:32.039007 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:32.039712 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:32.039722 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:32.040168 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 22:48:32.040506 (+ 338us) client_negotiation.cc:240] Negotiation successful
0807 22:48:32.041192 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":502,"threads_started":1}
I20260807 22:48:32.045099 8249 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46515
I20260807 22:48:32.045493 8249 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:32.046422 8249 heartbeater.cc:511] Master 127.20.10.190:46515 requested a full tablet report, sending...
I20260807 22:48:32.049454 8039 ts_manager.cc:194] Registered new tserver with Master: 476901e77abf4b3fa3f2ca3186a191d9 (127.20.10.129:33099)
I20260807 22:48:32.052153 8039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:33061
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:48:32.116755 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:32.520890 8255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:32.521246 8255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:32.521447 8255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:32.530431 8255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:32.530577 8255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:32.530809 8255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:32.542448 8255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:32.544330 8255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:32.545849 8255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:32.561815 8266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:32.561990 8262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:32.563071 8261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:32.563045 8255 server_base.cc:1034] running on GCE node
I20260807 22:48:32.564365 8255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:32.565798 8255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:32.567176 8255 hybrid_clock.cc:648] HybridClock initialized: now 1786142912566983 us; error 174 us; skew 500 ppm
I20260807 22:48:32.567564 8255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:32 dist-test-slave-csg5 krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142912, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:32.571589 8255 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:48:32.573745 8255 webserver.cc:460] Webserver started at http://127.20.10.130:35119/ using document root <none> and password file <none>
I20260807 22:48:32.574550 8255 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:32.574852 8255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:32.575235 8255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:32.578635 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "5b479a4df63d4acba7d9b8079a65e8fc"
format_stamp: "Formatted at 2026-08-07 22:48:32 on dist-test-slave-csg5"
server_key: "ce537e3d34234e981d85016befe46ee3"
server_key_iv: "e870f54b13aca62a2a6988e9e0fbec4c"
server_key_version: "encryptionkey@0"
I20260807 22:48:32.579689 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "5b479a4df63d4acba7d9b8079a65e8fc"
format_stamp: "Formatted at 2026-08-07 22:48:32 on dist-test-slave-csg5"
server_key: "ce537e3d34234e981d85016befe46ee3"
server_key_iv: "e870f54b13aca62a2a6988e9e0fbec4c"
server_key_version: "encryptionkey@0"
I20260807 22:48:32.587565 8255 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 22:48:32.592263 8273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:32.594303 8255 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:48:32.594590 8255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "5b479a4df63d4acba7d9b8079a65e8fc"
format_stamp: "Formatted at 2026-08-07 22:48:32 on dist-test-slave-csg5"
server_key: "ce537e3d34234e981d85016befe46ee3"
server_key_iv: "e870f54b13aca62a2a6988e9e0fbec4c"
server_key_version: "encryptionkey@0"
I20260807 22:48:32.594969 8255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:32.617998 8255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:32.624300 8255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:32.624734 8255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:32.626004 8255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:32.627990 8255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:32.628137 8255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:32.628348 8255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:32.628435 8255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:32.674386 8255 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:38023
I20260807 22:48:32.674495 8401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:38023 every 8 connection(s)
I20260807 22:48:32.675988 8255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:32.678081 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8255
I20260807 22:48:32.678321 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:32.679013 20522 external_mini_cluster.cc:1428] Setting key e47954171e0964b237af2b41c5ce44c9
Aug 07 22:48:32 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142912, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:32.703583 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:32.679371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:50655 (local address 127.20.10.190:46515)
0807 22:48:32.680005 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:32.680012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:32.682796 (+ 2784us) server_negotiation.cc:407] Connection header received
0807 22:48:32.683712 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:32.683717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:32.683828 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:32.683962 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:32.685899 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.686836 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.688623 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.689111 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.689268 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:32.694126 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:32.694132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:32.694136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:32.694179 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:32.698871 (+ 4692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:32.700188 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:32.700194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:32.700197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:32.700367 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:32.701084 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:32.701089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:32.701092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:32.701435 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:32.701592 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:32.702910 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0807 22:48:32.703267 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":300,"threads_started":1}
I20260807 22:48:32.705114 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:32.680127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46515 (local address 127.20.10.130:50655)
0807 22:48:32.681175 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:32.681285 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:32.683363 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:32.684309 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:32.684319 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:32.684986 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:32.685671 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:32.685692 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.687051 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.687055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:32.688425 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:32.688439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:32.689391 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:32.689399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:32.689668 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:32.690573 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:32.690607 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:32.693766 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:32.699115 (+ 5349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:32.699130 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:32.699154 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:32.700004 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:32.700578 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:32.700586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:32.700601 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:32.700912 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:32.701601 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:32.701614 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:32.702234 (+ 620us) client_negotiation.cc:769] Sending connection context
0807 22:48:32.702607 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 22:48:32.703329 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":433,"threads_started":1}
I20260807 22:48:32.707347 8402 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46515
I20260807 22:48:32.707732 8402 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:32.708859 8402 heartbeater.cc:511] Master 127.20.10.190:46515 requested a full tablet report, sending...
I20260807 22:48:32.711115 8039 ts_manager.cc:194] Registered new tserver with Master: 5b479a4df63d4acba7d9b8079a65e8fc (127.20.10.130:38023)
I20260807 22:48:32.712195 8039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:50655
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:48:32.768074 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:48:33.057225 8249 heartbeater.cc:503] Master 127.20.10.190:46515 was elected leader, sending a full tablet report...
W20260807 22:48:33.175917 8411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:33.176349 8411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:33.176477 8411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:33.185521 8411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:33.185678 8411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:33.185925 8411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:33.197259 8411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:33.198707 8411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:33.200624 8411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:33.216871 8420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:33.216848 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:33.216929 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:33.217865 8411 server_base.cc:1034] running on GCE node
I20260807 22:48:33.218590 8411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:33.219769 8411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:33.221026 8411 hybrid_clock.cc:648] HybridClock initialized: now 1786142913220977 us; error 35 us; skew 500 ppm
I20260807 22:48:33.221446 8411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:33 dist-test-slave-csg5 krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142913, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:33.225453 8411 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:48:33.227445 8411 webserver.cc:460] Webserver started at http://127.20.10.131:39713/ using document root <none> and password file <none>
I20260807 22:48:33.228369 8411 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:33.228546 8411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:33.228938 8411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:33.232378 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "13a75ef7a9c64dbd8d3630c228b57914"
format_stamp: "Formatted at 2026-08-07 22:48:33 on dist-test-slave-csg5"
server_key: "c599f509db45912886a45ef4c03a3d55"
server_key_iv: "82e099e4e6b8b539a1bf0625a30a5c37"
server_key_version: "encryptionkey@0"
I20260807 22:48:33.233304 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "13a75ef7a9c64dbd8d3630c228b57914"
format_stamp: "Formatted at 2026-08-07 22:48:33 on dist-test-slave-csg5"
server_key: "c599f509db45912886a45ef4c03a3d55"
server_key_iv: "82e099e4e6b8b539a1bf0625a30a5c37"
server_key_version: "encryptionkey@0"
I20260807 22:48:33.240495 8411 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 22:48:33.245404 8428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:33.247282 8411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:48:33.247486 8411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "13a75ef7a9c64dbd8d3630c228b57914"
format_stamp: "Formatted at 2026-08-07 22:48:33 on dist-test-slave-csg5"
server_key: "c599f509db45912886a45ef4c03a3d55"
server_key_iv: "82e099e4e6b8b539a1bf0625a30a5c37"
server_key_version: "encryptionkey@0"
I20260807 22:48:33.247715 8411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:33.270130 8411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:33.275496 8411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:33.275769 8411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:33.276847 8411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:33.278234 8411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:33.278335 8411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:33.278434 8411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:33.278479 8411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:33.315656 8411 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:43113
I20260807 22:48:33.315706 8556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:43113 every 8 connection(s)
I20260807 22:48:33.317126 8411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:33.320104 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8411
I20260807 22:48:33.320448 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:33.321035 20522 external_mini_cluster.cc:1428] Setting key efb3df23f16fbb02ac8e74deea10177f
Aug 07 22:48:33 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142913, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:33.341508 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.320330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:57549 (local address 127.20.10.190:46515)
0807 22:48:33.320951 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.320958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.322057 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 22:48:33.324172 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.324177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.324247 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.324341 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:33.326028 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.327163 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.329220 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.329650 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.329757 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.334177 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:33.334184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:33.334189 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:33.334238 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:33.337614 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.338607 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.338615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.338619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.338855 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.339379 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.339386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.339390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.339733 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:33.339937 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.340757 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.341033 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":314,"threads_started":1}
I20260807 22:48:33.342610 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.320949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:46515 (local address 127.20.10.131:57549)
0807 22:48:33.321896 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.321943 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.323912 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.324570 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.324578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.325230 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:33.325835 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.325853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.327444 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.327449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.329064 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.329079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.329952 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.329959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.330228 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.331082 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:33.331116 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:33.333850 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:33.337825 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:33.337837 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:33.337867 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:33.338454 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:33.338992 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:33.338997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:33.339001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:33.339230 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:33.339903 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:33.339912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:33.340345 (+ 433us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.340665 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.341319 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":383,"threads_started":1}
I20260807 22:48:33.344564 8557 heartbeater.cc:347] Connected to a master server at 127.20.10.190:46515
I20260807 22:48:33.344974 8557 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:33.345752 8557 heartbeater.cc:511] Master 127.20.10.190:46515 requested a full tablet report, sending...
I20260807 22:48:33.347590 8039 ts_manager.cc:194] Registered new tserver with Master: 13a75ef7a9c64dbd8d3630c228b57914 (127.20.10.131:43113)
I20260807 22:48:33.348685 8039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:57549
I20260807 22:48:33.349949 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:33.370214 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.354607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.20.10.190:46515)
0807 22:48:33.354879 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.354886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.355318 (+ 432us) server_negotiation.cc:407] Connection header received
0807 22:48:33.355538 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.355544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.355632 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.355747 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:33.357255 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.358031 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.359492 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.359957 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.360103 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.363378 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:33.363384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:33.363387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:33.363428 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:33.366845 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.367843 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.367849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.367853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.367995 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.368530 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.368534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.368536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.368889 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:33.369200 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.369632 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.369867 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:48:33.375411 8039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:33.383607 8039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:33.413950 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.400095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38023 (local address 127.0.0.1:34886)
0807 22:48:33.401058 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.401116 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.401319 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.402304 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.402314 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.402348 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:33.402930 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.402948 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.406135 (+ 3187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.406145 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.409557 (+ 3412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.409588 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.412837 (+ 3249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.412846 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.413072 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.413131 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.413227 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.413440 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":509,"threads_started":1}
I20260807 22:48:33.414940 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.400280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.20.10.130:38023)
0807 22:48:33.401509 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.401523 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.401576 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:48:33.401695 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.401704 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.402018 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.403181 (+ 1163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:33.403500 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.405864 (+ 2364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.409897 (+ 4033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.412535 (+ 2638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.413882 (+ 1347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.414155 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.414272 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.414483 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":917,"threads_started":1}
I20260807 22:48:33.419993 8327 tablet_service.cc:1467] Processing CreateTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc (DEFAULT_TABLE table=test-table [id=e06052c90bf444fc8f719a1587e89d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:33.421123 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.402071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.20.10.129:33099)
0807 22:48:33.412677 (+ 10606us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.412689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.412724 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:48:33.412888 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.412901 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.413209 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.413478 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:33.414565 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.416161 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.418314 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.419765 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.419922 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.420191 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.420531 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.420728 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10399,"thread_start_us":9912,"threads_started":1}
I20260807 22:48:33.421981 8327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14b4c7dec074453b6ee7f68d0b0d0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:33.422091 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.401634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33099 (local address 127.0.0.1:36026)
0807 22:48:33.402260 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.402278 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.402367 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.413472 (+ 11105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.413476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.413504 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:33.413899 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.413916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.416338 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.416343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.418111 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.418128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.420077 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.420085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.420287 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.420331 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.420452 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.421552 (+ 1100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":104,"thread_start_us":326,"threads_started":1}
I20260807 22:48:33.422580 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.402326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:43113 (local address 127.0.0.1:40844)
0807 22:48:33.410869 (+ 8543us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.410896 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.411072 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.412650 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.412659 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.412702 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:33.413296 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.413314 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.415677 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.415683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.418901 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.418921 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.421080 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.421087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.421921 (+ 834us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.421967 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.422077 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.422241 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8258,"thread_start_us":6673,"threads_started":1}
I20260807 22:48:33.423525 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.402963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.20.10.131:43113)
0807 22:48:33.410122 (+ 7159us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.410132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.411762 (+ 1630us) server_negotiation.cc:407] Connection header received
0807 22:48:33.411951 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.411960 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.412300 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.412569 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:33.413603 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.415479 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.419132 (+ 3653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.420691 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.420832 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.421715 (+ 883us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.422957 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.423233 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6915,"thread_start_us":7254,"threads_started":1}
I20260807 22:48:33.427834 8486 tablet_service.cc:1467] Processing CreateTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc (DEFAULT_TABLE table=test-table [id=e06052c90bf444fc8f719a1587e89d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:33.427212 8179 tablet_service.cc:1467] Processing CreateTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc (DEFAULT_TABLE table=test-table [id=e06052c90bf444fc8f719a1587e89d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:33.429651 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14b4c7dec074453b6ee7f68d0b0d0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:33.429780 8179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14b4c7dec074453b6ee7f68d0b0d0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:33.440593 8580 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Bootstrap starting.
I20260807 22:48:33.444898 8580 tablet_bootstrap.cc:654] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:33.444998 8581 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Bootstrap starting.
I20260807 22:48:33.446617 8580 log.cc:824] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:33.450389 8581 tablet_bootstrap.cc:654] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:33.452008 8582 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Bootstrap starting.
I20260807 22:48:33.452502 8581 log.cc:824] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:33.455993 8580 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: No bootstrap required, opened a new log
I20260807 22:48:33.456339 8580 ts_tablet_manager.cc:1397] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260807 22:48:33.456401 8582 tablet_bootstrap.cc:654] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:33.457815 8582 log.cc:824] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:33.461699 8582 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: No bootstrap required, opened a new log
I20260807 22:48:33.462045 8582 ts_tablet_manager.cc:1397] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 22:48:33.464150 8581 tablet_bootstrap.cc:492] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: No bootstrap required, opened a new log
I20260807 22:48:33.464469 8581 ts_tablet_manager.cc:1397] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.007s
I20260807 22:48:33.468531 8580 raft_consensus.cc:348] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.469142 8580 raft_consensus.cc:374] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:33.469271 8580 raft_consensus.cc:729] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b479a4df63d4acba7d9b8079a65e8fc, State: Initialized, Role: FOLLOWER
I20260807 22:48:33.470136 8580 consensus_queue.cc:260] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.473248 8402 heartbeater.cc:503] Master 127.20.10.190:46515 was elected leader, sending a full tablet report...
I20260807 22:48:33.474192 8580 ts_tablet_manager.cc:1428] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Time spent starting tablet: real 0.018s user 0.008s sys 0.011s
I20260807 22:48:33.474797 8581 raft_consensus.cc:348] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.475046 8582 raft_consensus.cc:348] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.475380 8581 raft_consensus.cc:374] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:33.475569 8582 raft_consensus.cc:374] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:33.475589 8581 raft_consensus.cc:729] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476901e77abf4b3fa3f2ca3186a191d9, State: Initialized, Role: FOLLOWER
I20260807 22:48:33.475704 8582 raft_consensus.cc:729] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a75ef7a9c64dbd8d3630c228b57914, State: Initialized, Role: FOLLOWER
I20260807 22:48:33.476303 8581 consensus_queue.cc:260] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.476388 8582 consensus_queue.cc:260] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.478365 8557 heartbeater.cc:503] Master 127.20.10.190:46515 was elected leader, sending a full tablet report...
I20260807 22:48:33.478462 8581 ts_tablet_manager.cc:1428] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 22:48:33.479110 8582 ts_tablet_manager.cc:1428] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
W20260807 22:48:33.517413 8250 tablet.cc:2369] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:33.569068 8559 tablet.cc:2369] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:33.678916 8403 tablet.cc:2369] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:33.720747 8590 raft_consensus.cc:482] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:33.721129 8590 raft_consensus.cc:504] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.723019 8590 leader_election.cc:288] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b479a4df63d4acba7d9b8079a65e8fc (127.20.10.130:38023), 476901e77abf4b3fa3f2ca3186a191d9 (127.20.10.129:33099)
I20260807 22:48:33.731717 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.723730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:33107 (local address 127.20.10.130:38023)
0807 22:48:33.723955 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.723975 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.724000 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:48:33.724111 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.724116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.724238 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.724369 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:33.725429 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.726482 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.729177 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.730617 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.730888 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.730981 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.731236 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.731426 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:48:33.731734 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.723414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:38023 (local address 127.20.10.131:33107)
0807 22:48:33.723641 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.723664 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.723814 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.724607 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.724614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.724655 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:33.725043 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.725060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.726708 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.726713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.728938 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.728965 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.730852 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.730857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.731028 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.731060 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.731162 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.731286 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":1}
I20260807 22:48:33.732511 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.724591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:42837 (local address 127.20.10.129:33099)
0807 22:48:33.724844 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.724849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.725051 (+ 202us) server_negotiation.cc:407] Connection header received
0807 22:48:33.725444 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.725449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.725545 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.725767 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:33.726374 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.727354 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.729726 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.731276 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.731461 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.731542 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.732032 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.732305 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:48:33.733266 8591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.723941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:33099 (local address 127.20.10.131:42837)
0807 22:48:33.724905 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:33.724928 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:33.725096 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:33.725787 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:33.725791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:33.725818 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:33.726165 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.726173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.727513 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.727518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.729542 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:33.729554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.731562 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.731568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:33.731787 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.731835 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:48:33.732815 (+ 980us) client_negotiation.cc:240] Negotiation successful
0807 22:48:33.732994 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":433,"threads_started":1}
I20260807 22:48:33.733084 8350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14b4c7dec074453b6ee7f68d0b0d0dc" candidate_uuid: "13a75ef7a9c64dbd8d3630c228b57914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" is_pre_election: true
I20260807 22:48:33.733734 8350 raft_consensus.cc:2454] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a75ef7a9c64dbd8d3630c228b57914 in term 0.
I20260807 22:48:33.734555 8431 leader_election.cc:302] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a75ef7a9c64dbd8d3630c228b57914, 5b479a4df63d4acba7d9b8079a65e8fc; no voters:
I20260807 22:48:33.734439 8201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14b4c7dec074453b6ee7f68d0b0d0dc" candidate_uuid: "13a75ef7a9c64dbd8d3630c228b57914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "476901e77abf4b3fa3f2ca3186a191d9" is_pre_election: true
I20260807 22:48:33.735114 8590 raft_consensus.cc:2781] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:33.735071 8201 raft_consensus.cc:2454] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a75ef7a9c64dbd8d3630c228b57914 in term 0.
I20260807 22:48:33.735307 8590 raft_consensus.cc:482] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:33.735430 8590 raft_consensus.cc:3037] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:33.738121 8590 raft_consensus.cc:504] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.738881 8590 leader_election.cc:288] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [CANDIDATE]: Term 1 election: Requested vote from peers 5b479a4df63d4acba7d9b8079a65e8fc (127.20.10.130:38023), 476901e77abf4b3fa3f2ca3186a191d9 (127.20.10.129:33099)
I20260807 22:48:33.739707 8201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14b4c7dec074453b6ee7f68d0b0d0dc" candidate_uuid: "13a75ef7a9c64dbd8d3630c228b57914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "476901e77abf4b3fa3f2ca3186a191d9"
I20260807 22:48:33.739706 8350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14b4c7dec074453b6ee7f68d0b0d0dc" candidate_uuid: "13a75ef7a9c64dbd8d3630c228b57914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b479a4df63d4acba7d9b8079a65e8fc"
I20260807 22:48:33.740001 8201 raft_consensus.cc:3037] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:33.740010 8350 raft_consensus.cc:3037] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:33.743968 8201 raft_consensus.cc:2454] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a75ef7a9c64dbd8d3630c228b57914 in term 1.
I20260807 22:48:33.743968 8350 raft_consensus.cc:2454] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a75ef7a9c64dbd8d3630c228b57914 in term 1.
I20260807 22:48:33.744649 8431 leader_election.cc:302] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a75ef7a9c64dbd8d3630c228b57914, 5b479a4df63d4acba7d9b8079a65e8fc; no voters:
I20260807 22:48:33.745039 8590 raft_consensus.cc:2781] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:33.745565 8590 raft_consensus.cc:686] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 1 LEADER]: Becoming Leader. State: Replica: 13a75ef7a9c64dbd8d3630c228b57914, State: Running, Role: LEADER
I20260807 22:48:33.746145 8590 consensus_queue.cc:237] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } }
I20260807 22:48:33.752437 8039 catalog_manager.cc:5495] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13a75ef7a9c64dbd8d3630c228b57914 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "13a75ef7a9c64dbd8d3630c228b57914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13a75ef7a9c64dbd8d3630c228b57914" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 43113 } health_report { overall_health: HEALTHY } } }
I20260807 22:48:33.796200 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.790280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.20.10.131:43113)
0807 22:48:33.790539 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.790546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.790571 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:48:33.790690 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.790694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.790787 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.790961 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:33.791408 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.792168 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.793648 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.794063 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.794227 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.794995 (+ 768us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:33.795539 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:33.795741 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.795903 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.795956 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:48:33.805526 8350 raft_consensus.cc:1261] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 1 FOLLOWER]: Refusing update from remote peer 13a75ef7a9c64dbd8d3630c228b57914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:33.805526 8201 raft_consensus.cc:1261] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 1 FOLLOWER]: Refusing update from remote peer 13a75ef7a9c64dbd8d3630c228b57914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:33.806784 8593 consensus_queue.cc:1035] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b479a4df63d4acba7d9b8079a65e8fc" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:33.807294 8590 consensus_queue.cc:1035] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "476901e77abf4b3fa3f2ca3186a191d9" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:33.822635 8597 mvcc.cc:205] Tried to move back new op lower bound from 7316041374935384064 to 7316041374709321728. Current Snapshot: MvccSnapshot[applied={T|T < 7316041374935384064}]
I20260807 22:48:33.836068 8039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:33.836318 8039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:33.840579 8039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1864ca27be44eebac67e1989b337810: Sending DeleteTablet for 3 replicas of tablet c14b4c7dec074453b6ee7f68d0b0d0dc
I20260807 22:48:33.842046 8327 tablet_service.cc:1514] Processing DeleteTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:33 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:34886
I20260807 22:48:33.842200 8179 tablet_service.cc:1514] Processing DeleteTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:33 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:36026
I20260807 22:48:33.843014 8486 tablet_service.cc:1514] Processing DeleteTablet for tablet c14b4c7dec074453b6ee7f68d0b0d0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:33 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:40844
I20260807 22:48:33.843480 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8107
I20260807 22:48:33.851325 8609 tablet_replica.cc:330] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: stopping tablet replica
I20260807 22:48:33.851893 8609 raft_consensus.cc:2229] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:33.852658 8609 raft_consensus.cc:2258] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:33.855331 8608 tablet_replica.cc:330] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: stopping tablet replica
I20260807 22:48:33.855860 8608 raft_consensus.cc:2229] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:33.856444 8608 raft_consensus.cc:2258] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:33.857425 8609 ts_tablet_manager.cc:1905] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:33.858955 8608 ts_tablet_manager.cc:1905] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:33.867532 8607 tablet_replica.cc:330] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: stopping tablet replica
I20260807 22:48:33.867777 8608 ts_tablet_manager.cc:1918] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 22:48:33 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:48:33.868024 8608 log.cc:1196] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/c14b4c7dec074453b6ee7f68d0b0d0dc
I20260807 22:48:33.868330 8607 raft_consensus.cc:2229] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:33.868666 8608 ts_tablet_manager.cc:1939] T c14b4c7dec074453b6ee7f68d0b0d0dc P 476901e77abf4b3fa3f2ca3186a191d9: Deleting consensus metadata
I20260807 22:48:33.868925 8609 ts_tablet_manager.cc:1918] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:33.869351 8609 log.cc:1196] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/c14b4c7dec074453b6ee7f68d0b0d0dc
I20260807 22:48:33.870160 8609 ts_tablet_manager.cc:1939] T c14b4c7dec074453b6ee7f68d0b0d0dc P 13a75ef7a9c64dbd8d3630c228b57914: Deleting consensus metadata
I20260807 22:48:33.871165 8023 catalog_manager.cc:4841] TS 476901e77abf4b3fa3f2ca3186a191d9 (127.20.10.129:33099): tablet c14b4c7dec074453b6ee7f68d0b0d0dc (table test-table [id=e06052c90bf444fc8f719a1587e89d32]) successfully deleted
I20260807 22:48:33.871822 8607 raft_consensus.cc:2258] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:33.873485 8023 catalog_manager.cc:4841] TS 13a75ef7a9c64dbd8d3630c228b57914 (127.20.10.131:43113): tablet c14b4c7dec074453b6ee7f68d0b0d0dc (table test-table [id=e06052c90bf444fc8f719a1587e89d32]) successfully deleted
I20260807 22:48:33.875082 8607 ts_tablet_manager.cc:1905] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:33.878484 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:33.844327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.20.10.129:33099)
0807 22:48:33.844518 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:33.844524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:33.855725 (+ 11201us) server_negotiation.cc:407] Connection header received
0807 22:48:33.856148 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:33.856153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:33.856240 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:33.856335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:33.858520 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.859673 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.861675 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:33.862343 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:33.862476 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:33.870152 (+ 7676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:33.870166 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:33.870186 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:33.870244 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:33.874720 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.875818 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.875828 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.875832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.876025 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:33.876550 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:33.876556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:33.876560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:33.877051 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:33.877383 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:33.877799 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 22:48:33.878064 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:48:33.879323 8244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:33.881973 8607 ts_tablet_manager.cc:1918] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:33.882193 8607 log.cc:1196] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/c14b4c7dec074453b6ee7f68d0b0d0dc
I20260807 22:48:33.882826 8607 ts_tablet_manager.cc:1939] T c14b4c7dec074453b6ee7f68d0b0d0dc P 5b479a4df63d4acba7d9b8079a65e8fc: Deleting consensus metadata
I20260807 22:48:33.885403 8023 catalog_manager.cc:4841] TS 5b479a4df63d4acba7d9b8079a65e8fc (127.20.10.130:38023): tablet c14b4c7dec074453b6ee7f68d0b0d0dc (table test-table [id=e06052c90bf444fc8f719a1587e89d32]) successfully deleted
I20260807 22:48:33.980917 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8107
I20260807 22:48:34.008118 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8255
Aug 07 22:48:34 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:48:34.028216 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:34.009045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.20.10.130:38023)
0807 22:48:34.009265 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:34.009272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:34.009295 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:48:34.009530 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:34.009535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:34.009605 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:34.009719 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:34.011172 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:34.011906 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:34.013251 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:34.013660 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:34.013819 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:34.019022 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:34.019050 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:34.019072 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:34.019134 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:34.023890 (+ 4756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:34.024875 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:34.024889 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:34.024893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:34.025116 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:34.025754 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:34.025760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:34.025764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:34.026346 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:34.026809 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:34.027489 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 22:48:34.027780 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:48:34.029174 8396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:34.127960 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8255
I20260807 22:48:34.154410 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8411
Aug 07 22:48:34 dist-test-slave-csg5 krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:48:34.176942 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:34.155597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.20.10.131:43113)
0807 22:48:34.155976 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:34.155984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:34.156223 (+ 239us) server_negotiation.cc:407] Connection header received
0807 22:48:34.156571 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:34.156577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:34.156669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:34.156848 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:34.158406 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:34.159468 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:34.160983 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:34.161626 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:34.161816 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:34.167918 (+ 6102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:34.167936 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:34.167962 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:34.168027 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:34.172804 (+ 4777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:34.173907 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:34.173918 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:34.173922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:34.174128 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:34.174793 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:34.174799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:34.174802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:34.175371 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:34.175774 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:34.176233 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:48:34.176510 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260807 22:48:34.178059 8551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:34.324254 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8411
I20260807 22:48:34.353000 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8004
I20260807 22:48:34.354389 8071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:34.473184 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8004
2026-08-07T22:48:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5459 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T22:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:34Z Disabled control of system clock
I20260807 22:48:34.583225 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44155
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:44155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:34.973667 8618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:34.974040 8618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:34.974267 8618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:34.983198 8618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:34.983343 8618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:34.983438 8618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:34.983544 8618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:34.983659 8618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:34.994379 8618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:44155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:34.995886 8618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:34.998039 8618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:35.012020 8623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:35.011981 8624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:35.012053 8626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:35.013696 8618 server_base.cc:1034] running on GCE node
I20260807 22:48:35.014879 8618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:35.016673 8618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:35.017968 8618 hybrid_clock.cc:648] HybridClock initialized: now 1786142915017920 us; error 42 us; skew 500 ppm
I20260807 22:48:35.018416 8618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:35.021704 8618 webserver.cc:460] Webserver started at http://127.20.10.190:42913/ using document root <none> and password file <none>
I20260807 22:48:35.022601 8618 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:35.022807 8618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:35.023209 8618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:35.026851 8618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "d9e74459ec77439c971650ec7b485306"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "136d6583f847eff652205af31f51088c"
server_key_iv: "4b439a7165a55c2e09bde4f3caeb0957"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.028064 8618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "d9e74459ec77439c971650ec7b485306"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "136d6583f847eff652205af31f51088c"
server_key_iv: "4b439a7165a55c2e09bde4f3caeb0957"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.034299 8618 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:48:35.038942 8632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:35.040736 8618 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 22:48:35.041049 8618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "d9e74459ec77439c971650ec7b485306"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "136d6583f847eff652205af31f51088c"
server_key_iv: "4b439a7165a55c2e09bde4f3caeb0957"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.041424 8618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:35.072963 8618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:35.074952 8618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:35.075419 8618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:35.104437 8618 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:44155
I20260807 22:48:35.104450 8686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:44155 every 8 connection(s)
I20260807 22:48:35.106823 8618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:35.113107 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8618
I20260807 22:48:35.113332 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:35.113353 8687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:35.113824 20522 external_mini_cluster.cc:1428] Setting key 39474fa9d26dc5dc780a70d9357b22a6
I20260807 22:48:35.132923 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:35.116017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.20.10.190:44155)
0807 22:48:35.117312 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:35.117345 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:35.117470 (+ 125us) server_negotiation.cc:407] Connection header received
0807 22:48:35.120908 (+ 3438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:35.120956 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:35.121636 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:35.122156 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:35.122936 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.124630 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.127055 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.128034 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.128275 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:35.129729 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:35.129739 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:35.129757 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:35.130509 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:35.130629 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:35.130720 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:35.130898 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:35.131156 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 22:48:35.131574 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":451,"threads_started":1}
I20260807 22:48:35.133886 8687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: Bootstrap starting.
I20260807 22:48:35.138329 8687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:35.139746 8687 log.cc:824] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:35.143558 8687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: No bootstrap required, opened a new log
I20260807 22:48:35.151947 8687 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } }
I20260807 22:48:35.152462 8687 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:35.152587 8687 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e74459ec77439c971650ec7b485306, State: Initialized, Role: FOLLOWER
I20260807 22:48:35.153234 8687 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } }
I20260807 22:48:35.153537 8687 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:35.153844 8687 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:35.154089 8687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:35.157171 8687 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } }
I20260807 22:48:35.157752 8687 leader_election.cc:302] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e74459ec77439c971650ec7b485306; no voters:
I20260807 22:48:35.158435 8687 leader_election.cc:288] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:35.158546 8692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:35.159607 8692 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [term 1 LEADER]: Becoming Leader. State: Replica: d9e74459ec77439c971650ec7b485306, State: Running, Role: LEADER
I20260807 22:48:35.160275 8692 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } }
I20260807 22:48:35.161235 8687 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:35.163450 8694 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e74459ec77439c971650ec7b485306. Latest consensus state: current_term: 1 leader_uuid: "d9e74459ec77439c971650ec7b485306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } } }
I20260807 22:48:35.164009 8694 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:35.164199 8693 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e74459ec77439c971650ec7b485306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e74459ec77439c971650ec7b485306" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44155 } } }
I20260807 22:48:35.164547 8693 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:35.167011 8697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:35.173630 8697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:35.184456 8697 catalog_manager.cc:1347] Generated new cluster ID: 8f2ab11dcb0f40a08f44af8a3d525ae0
I20260807 22:48:35.184604 8697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:35.206672 8697 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:35.208704 8697 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:35.220937 8697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: Generated new TSK 0
I20260807 22:48:35.222046 8697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:35.239346 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--builtin_ntp_servers=127.20.10.148:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:35.639703 8712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:35.640038 8712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:35.640209 8712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:35.648314 8712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:35.648438 8712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:35.648654 8712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:35.658460 8712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:35.660044 8712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:35.662156 8712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:35.678635 8719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:35.678575 8721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:35.679064 8718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:35.679141 8712 server_base.cc:1034] running on GCE node
I20260807 22:48:35.680058 8712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:35.681413 8712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:35.682814 8712 hybrid_clock.cc:648] HybridClock initialized: now 1786142915682756 us; error 51 us; skew 500 ppm
I20260807 22:48:35.683287 8712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:35.686471 8712 webserver.cc:460] Webserver started at http://127.20.10.129:32769/ using document root <none> and password file <none>
I20260807 22:48:35.687462 8712 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:35.687638 8712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:35.688035 8712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:35.691821 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "6672ce6ed2424340833b9e9b27a2be08"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "323e02b411351f01afd0557fa7cab10e"
server_key_iv: "e263c9889e46ba5bce7a5c4639ff379c"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.692951 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "6672ce6ed2424340833b9e9b27a2be08"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "323e02b411351f01afd0557fa7cab10e"
server_key_iv: "e263c9889e46ba5bce7a5c4639ff379c"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.699843 8712 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 22:48:35.704679 8728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:35.706736 8712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:48:35.707026 8712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "6672ce6ed2424340833b9e9b27a2be08"
format_stamp: "Formatted at 2026-08-07 22:48:35 on dist-test-slave-csg5"
server_key: "323e02b411351f01afd0557fa7cab10e"
server_key_iv: "e263c9889e46ba5bce7a5c4639ff379c"
server_key_version: "encryptionkey@0"
I20260807 22:48:35.707337 8712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:35.720440 8712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:35.721426 8712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:35.721771 8712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:35.723006 8712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:35.724844 8712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:35.724982 8712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:35.725152 8712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:35.725245 8712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:35.767745 8712 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:39767
I20260807 22:48:35.767776 8851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:39767 every 8 connection(s)
I20260807 22:48:35.769428 8712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:35.771967 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8712
I20260807 22:48:35.772217 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:35.772881 20522 external_mini_cluster.cc:1428] Setting key 1814289e3b1f352b85fa7f558de09b24
I20260807 22:48:35.776925 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--builtin_ntp_servers=127.20.10.148:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:35.787860 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:35.772769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:39255 (local address 127.20.10.190:44155)
0807 22:48:35.773288 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:35.773292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:35.774975 (+ 1683us) server_negotiation.cc:407] Connection header received
0807 22:48:35.777821 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:35.777826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:35.777906 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:35.778030 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:35.779696 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.780730 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.783985 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.784582 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.784723 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:35.786872 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:35.786878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:35.786882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:35.786907 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:35.786957 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:35.786970 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:35.787052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:35.787405 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:48:35.787515 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260807 22:48:35.789386 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:35.773638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44155 (local address 127.20.10.129:39255)
0807 22:48:35.774791 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:35.774851 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:35.777406 (+ 2555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:35.778339 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:35.778353 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:35.778499 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:35.779457 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:35.779488 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.780962 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.780972 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:35.783744 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:35.783771 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:35.784899 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:35.784910 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:35.785316 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:35.786284 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:35.786290 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:35.786367 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:35.786502 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:35.787085 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:35.787099 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:35.787111 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:48:35.787321 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 22:48:35.787822 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":376,"threads_started":1}
I20260807 22:48:35.791932 8852 heartbeater.cc:347] Connected to a master server at 127.20.10.190:44155
I20260807 22:48:35.792423 8852 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:35.793363 8852 heartbeater.cc:511] Master 127.20.10.190:44155 requested a full tablet report, sending...
I20260807 22:48:35.795980 8651 ts_manager.cc:194] Registered new tserver with Master: 6672ce6ed2424340833b9e9b27a2be08 (127.20.10.129:39767)
I20260807 22:48:35.797761 8651 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.129:39255
W20260807 22:48:36.170883 8857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:36.171209 8857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:36.171375 8857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:36.179332 8857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:36.179461 8857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:36.179638 8857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:36.189852 8857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:36.191454 8857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:36.193503 8857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:36.211153 8865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:36.210992 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:36.211557 8866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:36.213109 8857 server_base.cc:1034] running on GCE node
I20260807 22:48:36.213996 8857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:36.215257 8857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:36.216542 8857 hybrid_clock.cc:648] HybridClock initialized: now 1786142916216473 us; error 62 us; skew 500 ppm
I20260807 22:48:36.216935 8857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:36.225121 8857 webserver.cc:460] Webserver started at http://127.20.10.130:38771/ using document root <none> and password file <none>
I20260807 22:48:36.226042 8857 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:36.226208 8857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:36.226588 8857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:36.230010 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "f42948960cdb44178adf5f7f22a51748"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "230790058361999f50b5f69d5bbb5461"
server_key_iv: "0143829143ae8e1a20831b808bf2cfdd"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.231077 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "f42948960cdb44178adf5f7f22a51748"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "230790058361999f50b5f69d5bbb5461"
server_key_iv: "0143829143ae8e1a20831b808bf2cfdd"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.237347 8857 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:48:36.241583 8876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.243505 8857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:48:36.243901 8857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "f42948960cdb44178adf5f7f22a51748"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "230790058361999f50b5f69d5bbb5461"
server_key_iv: "0143829143ae8e1a20831b808bf2cfdd"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.244256 8857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:36.267876 8857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:36.268878 8857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:36.269225 8857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:36.270318 8857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:36.272234 8857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:36.272375 8857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.272516 8857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:36.272607 8857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.308734 8998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:34091 every 8 connection(s)
I20260807 22:48:36.308734 8857 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:34091
I20260807 22:48:36.310365 8857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:36.319442 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 8857
I20260807 22:48:36.319656 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:36.320268 20522 external_mini_cluster.cc:1428] Setting key 092dba2fa94bb3b57a9fdcb771917e4b
I20260807 22:48:36.323976 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--builtin_ntp_servers=127.20.10.148:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:36.325680 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.313727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:50147 (local address 127.20.10.190:44155)
0807 22:48:36.314221 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.314226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.315543 (+ 1317us) server_negotiation.cc:407] Connection header received
0807 22:48:36.317517 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.317523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.317630 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.317759 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:36.319333 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.320269 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.322245 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.322777 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.322926 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.324716 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:36.324722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:36.324725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:36.324744 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:36.324787 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:36.324800 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:36.324917 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.325178 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.325293 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260807 22:48:36.326884 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.314374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44155 (local address 127.20.10.130:50147)
0807 22:48:36.315369 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:36.315414 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:36.317206 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:36.318027 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:36.318036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:36.318130 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:36.318852 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.318873 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.320510 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.320516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.322043 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.322061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.323045 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.323053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.323323 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.324299 (+ 976us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:36.324303 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:36.324367 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:36.324483 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:36.324916 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:36.324932 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:36.324943 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:48:36.325168 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 22:48:36.325631 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":343,"threads_started":1}
I20260807 22:48:36.328583 8999 heartbeater.cc:347] Connected to a master server at 127.20.10.190:44155
I20260807 22:48:36.328998 8999 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:36.330099 8999 heartbeater.cc:511] Master 127.20.10.190:44155 requested a full tablet report, sending...
I20260807 22:48:36.332119 8651 ts_manager.cc:194] Registered new tserver with Master: f42948960cdb44178adf5f7f22a51748 (127.20.10.130:34091)
I20260807 22:48:36.333563 8651 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.130:50147
W20260807 22:48:36.723835 9003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:36.724354 9003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:36.724501 9003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:36.732477 9003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:36.732614 9003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:36.732735 9003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:36.742600 9003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:44155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:36.744094 9003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:36.746443 9003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:36.762344 9009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:36.762372 9008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:36.762286 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:36.763228 9003 server_base.cc:1034] running on GCE node
I20260807 22:48:36.763926 9003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:36.764990 9003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:36.766346 9003 hybrid_clock.cc:648] HybridClock initialized: now 1786142916766280 us; error 57 us; skew 500 ppm
I20260807 22:48:36.766847 9003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:36.769995 9003 webserver.cc:460] Webserver started at http://127.20.10.131:43515/ using document root <none> and password file <none>
I20260807 22:48:36.770932 9003 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:36.771085 9003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:36.771442 9003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:36.774753 9003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "f861d8e5634e48d5a627dd4c4a65f592"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "2b4656550e24a97746d89162f394a607"
server_key_iv: "72b598bb3c673a1dbc05f45299b939a2"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.775684 9003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "f861d8e5634e48d5a627dd4c4a65f592"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "2b4656550e24a97746d89162f394a607"
server_key_iv: "72b598bb3c673a1dbc05f45299b939a2"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.781558 9003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:48:36.785815 9020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.787495 9003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:48:36.787714 9003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "f861d8e5634e48d5a627dd4c4a65f592"
format_stamp: "Formatted at 2026-08-07 22:48:36 on dist-test-slave-csg5"
server_key: "2b4656550e24a97746d89162f394a607"
server_key_iv: "72b598bb3c673a1dbc05f45299b939a2"
server_key_version: "encryptionkey@0"
I20260807 22:48:36.787981 9003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:36.801671 8852 heartbeater.cc:503] Master 127.20.10.190:44155 was elected leader, sending a full tablet report...
I20260807 22:48:36.809306 9003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:36.810348 9003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:36.810792 9003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:36.812254 9003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:36.813964 9003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:36.814110 9003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.814271 9003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:36.814370 9003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:36.851996 9003 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:35985
I20260807 22:48:36.852093 9143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:35985 every 8 connection(s)
I20260807 22:48:36.853400 9003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:36.857394 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9003
I20260807 22:48:36.857615 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:36.858232 20522 external_mini_cluster.cc:1428] Setting key 016c7c7f240e835d6cf2bb48d9be8c2d
I20260807 22:48:36.867436 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.856226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:33619 (local address 127.20.10.190:44155)
0807 22:48:36.856716 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.856721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.858220 (+ 1499us) server_negotiation.cc:407] Connection header received
0807 22:48:36.860506 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.860511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.860612 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.860732 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:36.862015 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.862963 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.864495 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.864909 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.865047 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.866466 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:36.866471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:36.866474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:36.866493 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:36.866536 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:36.866547 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:36.866650 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.866915 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.867020 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":242,"threads_started":1}
I20260807 22:48:36.868494 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.856834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44155 (local address 127.20.10.131:33619)
0807 22:48:36.858041 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:36.858085 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:36.860240 (+ 2155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:36.860976 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:36.860984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:36.861104 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:36.861851 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.861870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.863115 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.863125 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.864352 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.864363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.865159 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.865167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.865412 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.866120 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:36.866122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:36.866173 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:36.866255 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:36.866669 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:36.866684 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:36.866695 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:48:36.866900 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 22:48:36.867342 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":381,"threads_started":1}
I20260807 22:48:36.870118 9144 heartbeater.cc:347] Connected to a master server at 127.20.10.190:44155
I20260807 22:48:36.870446 9144 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:36.871268 9144 heartbeater.cc:511] Master 127.20.10.190:44155 requested a full tablet report, sending...
I20260807 22:48:36.872975 8650 ts_manager.cc:194] Registered new tserver with Master: f861d8e5634e48d5a627dd4c4a65f592 (127.20.10.131:35985)
I20260807 22:48:36.874017 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:36.874188 8650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.131:33619
I20260807 22:48:36.885401 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.878594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.20.10.190:44155)
0807 22:48:36.878849 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.878854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.879372 (+ 518us) server_negotiation.cc:407] Connection header received
0807 22:48:36.879515 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.879519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.879587 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.879677 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:36.880256 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.880979 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.882513 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.883061 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.883165 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.884492 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:36.884497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:36.884500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:36.884521 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:36.884561 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:36.884574 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:36.884738 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.885018 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.885073 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260807 22:48:36.890213 8650 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:36.897847 8650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:36.925297 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.912570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:34091 (local address 127.0.0.1:51688)
0807 22:48:36.915214 (+ 2644us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:36.915231 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:36.915350 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:36.916022 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:36.916029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:36.916066 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:36.916557 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.916568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.918816 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.918822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.921612 (+ 2790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.921629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.924084 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.924091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.924364 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.924442 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 22:48:36.924590 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:48:36.924819 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2522,"mutex_wait_us":6,"spinlock_wait_cycles":7424,"thread_start_us":1335,"threads_started":1}
I20260807 22:48:36.925292 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.913041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.20.10.130:34091)
0807 22:48:36.914890 (+ 1849us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.914902 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.915465 (+ 563us) server_negotiation.cc:407] Connection header received
0807 22:48:36.915565 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.915574 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.915810 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.916756 (+ 946us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:36.916976 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.918621 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.921847 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.923457 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.923620 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.923933 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.924659 (+ 726us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.924869 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668,"thread_start_us":1693,"threads_started":1}
I20260807 22:48:36.925884 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.912614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.20.10.131:35985)
0807 22:48:36.915082 (+ 2468us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.915093 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.915138 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:48:36.915261 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.915273 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.915550 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.916680 (+ 1130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:36.916979 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.918610 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.922496 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.923908 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.924048 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.924333 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.925293 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.925485 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1123,"thread_start_us":950,"threads_started":1}
I20260807 22:48:36.926487 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.912374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:35985 (local address 127.0.0.1:55926)
0807 22:48:36.914383 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:36.914429 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:36.914679 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:36.915800 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:36.915808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:36.915848 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:36.916443 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.916459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.918816 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.918822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.922287 (+ 3465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.922299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.924293 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.924303 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.925020 (+ 717us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.925059 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:48:36.925885 (+ 826us) client_negotiation.cc:240] Negotiation successful
0807 22:48:36.926063 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1734,"mutex_wait_us":489,"spinlock_wait_cycles":2816,"thread_start_us":2566,"threads_started":1}
I20260807 22:48:36.929121 9161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.915394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:39767 (local address 127.0.0.1:45896)
0807 22:48:36.919760 (+ 4366us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:36.919779 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:36.919895 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:36.921214 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:36.921224 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:36.921255 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:36.921708 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.921723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.923713 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.923718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.926169 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:36.926186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.928188 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.928195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:36.928368 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.928401 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:48:36.928473 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:48:36.928614 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4213,"mutex_wait_us":2,"thread_start_us":4042,"threads_started":1}
I20260807 22:48:36.929942 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:36.915666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.20.10.129:39767)
0807 22:48:36.920558 (+ 4892us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:36.920567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:36.920596 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:48:36.920691 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:36.920700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:36.920973 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:36.921193 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:36.921924 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.923550 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.926424 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:36.927953 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:36.929074 (+ 1121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:36.929427 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:36.929511 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 22:48:36.929687 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4701,"thread_start_us":4538,"threads_started":1}
I20260807 22:48:36.930436 8929 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 (DEFAULT_TABLE table=test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:36.930435 9073 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 (DEFAULT_TABLE table=test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:36.932081 9073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf7a5fa6db1426d8426238a5969d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:36.932081 8929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf7a5fa6db1426d8426238a5969d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:36.933476 8781 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 (DEFAULT_TABLE table=test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:36.934847 8781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf7a5fa6db1426d8426238a5969d996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:36.947899 9163 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Bootstrap starting.
I20260807 22:48:36.950312 9164 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Bootstrap starting.
I20260807 22:48:36.951308 9166 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Bootstrap starting.
I20260807 22:48:36.953537 9163 tablet_bootstrap.cc:654] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:36.955071 9163 log.cc:824] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:36.955314 9164 tablet_bootstrap.cc:654] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:36.955444 9166 tablet_bootstrap.cc:654] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:36.956563 9164 log.cc:824] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:36.957057 9166 log.cc:824] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:36.959880 9163 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: No bootstrap required, opened a new log
I20260807 22:48:36.960240 9163 ts_tablet_manager.cc:1397] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260807 22:48:36.960256 9164 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: No bootstrap required, opened a new log
I20260807 22:48:36.960310 9166 tablet_bootstrap.cc:492] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: No bootstrap required, opened a new log
I20260807 22:48:36.960557 9164 ts_tablet_manager.cc:1397] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 22:48:36.960568 9166 ts_tablet_manager.cc:1397] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 22:48:36.969053 9164 raft_consensus.cc:348] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.969542 9164 raft_consensus.cc:374] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:36.969712 9164 raft_consensus.cc:729] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42948960cdb44178adf5f7f22a51748, State: Initialized, Role: FOLLOWER
I20260807 22:48:36.970487 9164 consensus_queue.cc:260] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.971940 8999 heartbeater.cc:503] Master 127.20.10.190:44155 was elected leader, sending a full tablet report...
I20260807 22:48:36.972132 9163 raft_consensus.cc:348] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.972625 9163 raft_consensus.cc:374] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:36.972131 9166 raft_consensus.cc:348] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.972756 9163 raft_consensus.cc:729] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6672ce6ed2424340833b9e9b27a2be08, State: Initialized, Role: FOLLOWER
I20260807 22:48:36.972818 9166 raft_consensus.cc:374] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:36.972872 9164 ts_tablet_manager.cc:1428] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260807 22:48:36.972911 9166 raft_consensus.cc:729] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f861d8e5634e48d5a627dd4c4a65f592, State: Initialized, Role: FOLLOWER
I20260807 22:48:36.973601 9166 consensus_queue.cc:260] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.973628 9163 consensus_queue.cc:260] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:36.981042 9144 heartbeater.cc:503] Master 127.20.10.190:44155 was elected leader, sending a full tablet report...
I20260807 22:48:36.982470 9163 ts_tablet_manager.cc:1428] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Time spent starting tablet: real 0.022s user 0.014s sys 0.007s
I20260807 22:48:36.982597 9166 ts_tablet_manager.cc:1428] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Time spent starting tablet: real 0.022s user 0.019s sys 0.004s
W20260807 22:48:37.022791 8853 tablet.cc:2369] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:37.047171 9172 raft_consensus.cc:482] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:37.047505 9172 raft_consensus.cc:504] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:37.049516 9172 leader_election.cc:288] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f42948960cdb44178adf5f7f22a51748 (127.20.10.130:34091), 6672ce6ed2424340833b9e9b27a2be08 (127.20.10.129:39767)
I20260807 22:48:37.057065 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.050205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:37857 (local address 127.20.10.130:34091)
0807 22:48:37.050400 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.050407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.050689 (+ 282us) server_negotiation.cc:407] Connection header received
0807 22:48:37.051008 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.051013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.051130 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.051224 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:37.051968 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.052811 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.054845 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.056076 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.056210 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.056266 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.056719 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.056839 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:48:37.057175 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.050158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:34091 (local address 127.20.10.131:37857)
0807 22:48:37.050513 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:37.050537 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:37.050772 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:37.051511 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:37.051525 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:37.051555 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:37.051762 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:37.051773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.052960 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.052965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:37.054607 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:37.054620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.056251 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.056258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:37.056485 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.056527 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 22:48:37.056692 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 22:48:37.056859 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":31}
I20260807 22:48:37.058111 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.050291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:39767 (local address 127.20.10.131:43697)
0807 22:48:37.051117 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:37.051132 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:37.051377 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:37.051791 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:37.051795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:37.051824 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:37.052220 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:37.052237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.053619 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.053624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:37.055517 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:37.055532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.057229 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.057237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:37.057476 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.057511 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:48:37.057684 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:48:37.057848 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":493,"threads_started":1}
I20260807 22:48:37.058146 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.050155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:43697 (local address 127.20.10.129:39767)
0807 22:48:37.050400 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.050407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.051238 (+ 831us) server_negotiation.cc:407] Connection header received
0807 22:48:37.051566 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.051572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.051669 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.051770 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:37.052456 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.053443 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.055778 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.057007 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.057212 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.057286 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.057749 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.057869 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:48:37.058531 8950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf7a5fa6db1426d8426238a5969d996" candidate_uuid: "f861d8e5634e48d5a627dd4c4a65f592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42948960cdb44178adf5f7f22a51748" is_pre_election: true
I20260807 22:48:37.059196 8950 raft_consensus.cc:2454] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f861d8e5634e48d5a627dd4c4a65f592 in term 0.
I20260807 22:48:37.059712 8802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf7a5fa6db1426d8426238a5969d996" candidate_uuid: "f861d8e5634e48d5a627dd4c4a65f592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6672ce6ed2424340833b9e9b27a2be08" is_pre_election: true
I20260807 22:48:37.060050 9025 leader_election.cc:302] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f42948960cdb44178adf5f7f22a51748, f861d8e5634e48d5a627dd4c4a65f592; no voters:
I20260807 22:48:37.060259 8802 raft_consensus.cc:2454] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f861d8e5634e48d5a627dd4c4a65f592 in term 0.
I20260807 22:48:37.060586 9172 raft_consensus.cc:2781] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:37.060746 9172 raft_consensus.cc:482] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:37.060879 9172 raft_consensus.cc:3037] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:48:37.062943 9000 tablet.cc:2369] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:37.063519 9172 raft_consensus.cc:504] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:37.064265 9172 leader_election.cc:288] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [CANDIDATE]: Term 1 election: Requested vote from peers f42948960cdb44178adf5f7f22a51748 (127.20.10.130:34091), 6672ce6ed2424340833b9e9b27a2be08 (127.20.10.129:39767)
I20260807 22:48:37.064826 8950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf7a5fa6db1426d8426238a5969d996" candidate_uuid: "f861d8e5634e48d5a627dd4c4a65f592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42948960cdb44178adf5f7f22a51748"
I20260807 22:48:37.064975 8802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf7a5fa6db1426d8426238a5969d996" candidate_uuid: "f861d8e5634e48d5a627dd4c4a65f592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6672ce6ed2424340833b9e9b27a2be08"
I20260807 22:48:37.065104 8950 raft_consensus.cc:3037] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:37.065230 8802 raft_consensus.cc:3037] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:37.068925 8950 raft_consensus.cc:2454] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f861d8e5634e48d5a627dd4c4a65f592 in term 1.
I20260807 22:48:37.068910 8802 raft_consensus.cc:2454] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f861d8e5634e48d5a627dd4c4a65f592 in term 1.
I20260807 22:48:37.069521 9021 leader_election.cc:302] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6672ce6ed2424340833b9e9b27a2be08, f861d8e5634e48d5a627dd4c4a65f592; no voters:
I20260807 22:48:37.069912 9172 raft_consensus.cc:2781] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:37.070376 9172 raft_consensus.cc:686] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 1 LEADER]: Becoming Leader. State: Replica: f861d8e5634e48d5a627dd4c4a65f592, State: Running, Role: LEADER
I20260807 22:48:37.071056 9172 consensus_queue.cc:237] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } }
I20260807 22:48:37.076707 8650 catalog_manager.cc:5495] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 reported cstate change: term changed from 0 to 1, leader changed from <none> to f861d8e5634e48d5a627dd4c4a65f592 (127.20.10.131). New cstate: current_term: 1 leader_uuid: "f861d8e5634e48d5a627dd4c4a65f592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f861d8e5634e48d5a627dd4c4a65f592" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 } health_report { overall_health: UNKNOWN } } }
I20260807 22:48:37.097640 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.090865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55938 (local address 127.20.10.131:35985)
0807 22:48:37.091133 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.091138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.091353 (+ 215us) server_negotiation.cc:407] Connection header received
0807 22:48:37.091459 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.091463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.091563 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.091714 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:37.092255 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.093259 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.095013 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.095481 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.095608 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.096312 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:37.096869 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:37.097015 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.097220 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.097316 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 22:48:37.105399 9145 tablet.cc:2369] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:37.107525 8802 raft_consensus.cc:1261] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 1 FOLLOWER]: Refusing update from remote peer f861d8e5634e48d5a627dd4c4a65f592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:37.107525 8950 raft_consensus.cc:1261] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 1 FOLLOWER]: Refusing update from remote peer f861d8e5634e48d5a627dd4c4a65f592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:37.108700 9172 consensus_queue.cc:1035] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6672ce6ed2424340833b9e9b27a2be08" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:37.109118 9176 consensus_queue.cc:1035] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42948960cdb44178adf5f7f22a51748" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:37.125918 9179 mvcc.cc:205] Tried to move back new op lower bound from 7316041388459405312 to 7316041388327358464. Current Snapshot: MvccSnapshot[applied={T|T < 7316041388459405312}]
I20260807 22:48:37.144146 8650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:37.144467 8650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:37.149118 8650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9e74459ec77439c971650ec7b485306: Sending DeleteTablet for 3 replicas of tablet 4bf7a5fa6db1426d8426238a5969d996
I20260807 22:48:37.150543 9073 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:37 UTC) from {username='slave'} at 127.0.0.1:55926
I20260807 22:48:37.150719 8929 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:37 UTC) from {username='slave'} at 127.0.0.1:51688
I20260807 22:48:37.151880 8781 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf7a5fa6db1426d8426238a5969d996 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:37 UTC) from {username='slave'} at 127.0.0.1:45896
I20260807 22:48:37.155191 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8712
I20260807 22:48:37.164708 9190 tablet_replica.cc:330] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: stopping tablet replica
I20260807 22:48:37.165624 9188 tablet_replica.cc:330] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: stopping tablet replica
I20260807 22:48:37.166541 9189 tablet_replica.cc:330] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: stopping tablet replica
I20260807 22:48:37.166915 9190 raft_consensus.cc:2229] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:37.167570 9189 raft_consensus.cc:2229] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:37.167662 9190 raft_consensus.cc:2258] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:37.167903 9188 raft_consensus.cc:2229] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:37.168514 9189 raft_consensus.cc:2258] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:37.169307 9188 raft_consensus.cc:2258] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:37.170490 9190 ts_tablet_manager.cc:1905] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:37.172279 9189 ts_tablet_manager.cc:1905] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:37.173729 9188 ts_tablet_manager.cc:1905] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:37.179100 9190 ts_tablet_manager.cc:1918] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:37.179402 9190 log.cc:1196] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/4bf7a5fa6db1426d8426238a5969d996
I20260807 22:48:37.180133 9190 ts_tablet_manager.cc:1939] T 4bf7a5fa6db1426d8426238a5969d996 P 6672ce6ed2424340833b9e9b27a2be08: Deleting consensus metadata
I20260807 22:48:37.182332 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.157370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.20.10.129:39767)
0807 22:48:37.157649 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.157659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.167502 (+ 9843us) server_negotiation.cc:407] Connection header received
0807 22:48:37.168103 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.168109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.168234 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.168319 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:37.169381 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.170548 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.173205 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.173873 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.173980 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.180227 (+ 6247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:37.180245 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:37.180266 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:37.181185 (+ 919us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:37.181278 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:37.181311 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:37.181433 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.181753 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.181898 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:48:37.182804 8634 catalog_manager.cc:4841] TS 6672ce6ed2424340833b9e9b27a2be08 (127.20.10.129:39767): tablet 4bf7a5fa6db1426d8426238a5969d996 (table test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]) successfully deleted
I20260807 22:48:37.182859 9189 ts_tablet_manager.cc:1918] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:37.183331 9189 log.cc:1196] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/4bf7a5fa6db1426d8426238a5969d996
I20260807 22:48:37.184015 9189 ts_tablet_manager.cc:1939] T 4bf7a5fa6db1426d8426238a5969d996 P f42948960cdb44178adf5f7f22a51748: Deleting consensus metadata
I20260807 22:48:37.184223 8847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:37.186534 8637 catalog_manager.cc:4841] TS f42948960cdb44178adf5f7f22a51748 (127.20.10.130:34091): tablet 4bf7a5fa6db1426d8426238a5969d996 (table test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]) successfully deleted
I20260807 22:48:37.189168 9188 ts_tablet_manager.cc:1918] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:37.189512 9188 log.cc:1196] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/4bf7a5fa6db1426d8426238a5969d996
I20260807 22:48:37.190553 9188 ts_tablet_manager.cc:1939] T 4bf7a5fa6db1426d8426238a5969d996 P f861d8e5634e48d5a627dd4c4a65f592: Deleting consensus metadata
I20260807 22:48:37.192705 8634 catalog_manager.cc:4841] TS f861d8e5634e48d5a627dd4c4a65f592 (127.20.10.131:35985): tablet 4bf7a5fa6db1426d8426238a5969d996 (table test-table [id=dfbf4e6e1b9d4298977f70be6566ffe8]) successfully deleted
I20260807 22:48:37.285055 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8712
I20260807 22:48:37.312817 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8857
I20260807 22:48:37.323354 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.313886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.20.10.130:34091)
0807 22:48:37.314164 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.314172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.314387 (+ 215us) server_negotiation.cc:407] Connection header received
0807 22:48:37.315065 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.315076 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.315235 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.315373 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:37.315881 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.316853 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.318414 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.319194 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.319356 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.320909 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:37.320926 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:37.320951 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:37.322067 (+ 1116us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:37.322179 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:37.322209 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:37.322301 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.322582 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.322796 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 22:48:37.324292 8994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:37.422443 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8857
I20260807 22:48:37.449489 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9003
I20260807 22:48:37.458218 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:37.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.20.10.131:35985)
0807 22:48:37.450814 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:37.450821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:37.450960 (+ 139us) server_negotiation.cc:407] Connection header received
0807 22:48:37.451217 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:37.451222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:37.451326 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:37.451459 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:37.451958 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.452886 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.454325 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:37.455119 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:37.455303 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:37.456562 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:37.456572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:37.456589 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:37.457291 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:37.457353 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:37.457375 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:37.457540 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:37.457717 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 22:48:37.457835 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:48:37.459153 9139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:37.592351 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9003
I20260807 22:48:37.616127 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 8618
I20260807 22:48:37.617406 8682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:37.732620 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 8618
2026-08-07T22:48:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3261 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:37 dist-test-slave-csg5 krb5kdc[9201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:37 dist-test-slave-csg5 krb5kdc[9201](info): set up 2 sockets
Aug 07 22:48:37 dist-test-slave-csg5 krb5kdc[9201](info): commencing operation
krb5kdc: starting...
W20260807 22:48:38.972540 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:39 dist-test-slave-csg5 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:48:39.354909 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:41603
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:39.751574 9217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:39.751915 9217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:39.752126 9217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:39.760069 9217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:39.760212 9217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:39.760293 9217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:39.760406 9217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:39.760480 9217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:39.771941 9217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:39.773412 9217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:39.774860 9217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:39.788607 9222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:39.788585 9223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:39.788556 9225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:39.789415 9217 server_base.cc:1034] running on GCE node
I20260807 22:48:39.790405 9217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:39.792217 9217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:39.793465 9217 hybrid_clock.cc:648] HybridClock initialized: now 1786142919793412 us; error 37 us; skew 500 ppm
I20260807 22:48:39.793818 9217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:39 dist-test-slave-csg5 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:39.797271 9217 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:48:39.798959 9217 webserver.cc:460] Webserver started at http://127.20.10.190:36535/ using document root <none> and password file <none>
I20260807 22:48:39.799664 9217 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:39.799782 9217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:39.800050 9217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:39.803474 9217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "56ca11ce7f874e079e8307a7bab46339"
format_stamp: "Formatted at 2026-08-07 22:48:39 on dist-test-slave-csg5"
server_key: "ab488873c9cc9f3b276be8e52bebb5ac"
server_key_iv: "8d6ca69b7d06433392dbb397f37de14e"
server_key_version: "encryptionkey@0"
I20260807 22:48:39.804251 9217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "56ca11ce7f874e079e8307a7bab46339"
format_stamp: "Formatted at 2026-08-07 22:48:39 on dist-test-slave-csg5"
server_key: "ab488873c9cc9f3b276be8e52bebb5ac"
server_key_iv: "8d6ca69b7d06433392dbb397f37de14e"
server_key_version: "encryptionkey@0"
I20260807 22:48:39.810055 9217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 22:48:39.814098 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:39.815881 9217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:48:39.816092 9217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "56ca11ce7f874e079e8307a7bab46339"
format_stamp: "Formatted at 2026-08-07 22:48:39 on dist-test-slave-csg5"
server_key: "ab488873c9cc9f3b276be8e52bebb5ac"
server_key_iv: "8d6ca69b7d06433392dbb397f37de14e"
server_key_version: "encryptionkey@0"
I20260807 22:48:39.816318 9217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:39.836724 9217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:39.841871 9217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:39.842206 9217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:39.863739 9217 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:41603
I20260807 22:48:39.863796 9288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:41603 every 8 connection(s)
I20260807 22:48:39.865311 9217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:39.869537 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:39.873858 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9217
I20260807 22:48:39.874089 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:39.874713 20522 external_mini_cluster.cc:1428] Setting key 8162a259e3e6b5110d41c2cf01c19f86
I20260807 22:48:39.882534 9289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: Bootstrap starting.
I20260807 22:48:39.886978 9289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:39.888448 9289 log.cc:824] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:39.893183 9289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: No bootstrap required, opened a new log
Aug 07 22:48:39 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:39.903167 9289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } }
I20260807 22:48:39.903771 9289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:39.903926 9289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ca11ce7f874e079e8307a7bab46339, State: Initialized, Role: FOLLOWER
I20260807 22:48:39.904858 9289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } }
I20260807 22:48:39.905259 9289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:39.905426 9289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:39.905660 9289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:39.906905 9292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:39.876971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.20.10.190:41603)
0807 22:48:39.878013 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:39.878027 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:39.878077 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:48:39.881234 (+ 3157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:39.881266 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:39.881813 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:39.882281 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:39.884472 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:39.886229 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:39.888385 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:39.889910 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:39.890234 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:39.896864 (+ 6630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:39.896875 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:39.896895 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:39.896960 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:39.900898 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:39.901883 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:39.901892 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:39.901899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:39.902123 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:39.902635 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:39.902639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:39.902642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:39.903376 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:39.903775 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:39.904433 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 22:48:39.905070 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":532,"threads_started":1}
I20260807 22:48:39.909619 9289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } }
I20260807 22:48:39.910259 9289 leader_election.cc:302] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56ca11ce7f874e079e8307a7bab46339; no voters:
I20260807 22:48:39.911154 9289 leader_election.cc:288] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:39.911357 9294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:39.912233 9294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [term 1 LEADER]: Becoming Leader. State: Replica: 56ca11ce7f874e079e8307a7bab46339, State: Running, Role: LEADER
I20260807 22:48:39.913007 9294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } }
I20260807 22:48:39.913678 9289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:39.916402 9295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56ca11ce7f874e079e8307a7bab46339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } } }
I20260807 22:48:39.916899 9295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:39.916697 9296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56ca11ce7f874e079e8307a7bab46339. Latest consensus state: current_term: 1 leader_uuid: "56ca11ce7f874e079e8307a7bab46339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ca11ce7f874e079e8307a7bab46339" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 41603 } } }
I20260807 22:48:39.917165 9296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:39.920164 9300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:39.926934 9300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:39.939242 9300 catalog_manager.cc:1347] Generated new cluster ID: 28945baf0ab3422d8992ff6dab336743
I20260807 22:48:39.939405 9300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:39.962723 9300 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:39.964764 9300 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:39.978644 9300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: Generated new TSK 0
I20260807 22:48:39.979710 9300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.10.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
Entry for principal kudu/127.20.10.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab.
I20260807 22:48:40.088048 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:40.482515 9315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:40.482955 9315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:40.483129 9315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:40.491384 9315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:40.491560 9315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:40.491751 9315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:40.502314 9315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.129.keytab
--principal=kudu/127.20.10.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:40.503788 9315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:40.505268 9315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:40.520058 9323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:40.520133 9320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:40.520146 9321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:40.524838 9315 server_base.cc:1034] running on GCE node
I20260807 22:48:40.525588 9315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:40.526793 9315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:40.528084 9315 hybrid_clock.cc:648] HybridClock initialized: now 1786142920528034 us; error 36 us; skew 500 ppm
I20260807 22:48:40.528519 9315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:40 dist-test-slave-csg5 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142920, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:40.532526 9315 init.cc:377] Logged in from keytab as kudu/127.20.10.129@KRBTEST.COM (short username kudu)
I20260807 22:48:40.534596 9315 webserver.cc:460] Webserver started at http://127.20.10.129:38711/ using document root <none> and password file <none>
I20260807 22:48:40.535552 9315 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:40.535742 9315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:40.536140 9315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:40.539855 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "8c40b7ecd39848d99a16a0f381c94a9e"
format_stamp: "Formatted at 2026-08-07 22:48:40 on dist-test-slave-csg5"
server_key: "daf2bae8f762a7c7128fe646f8d86897"
server_key_iv: "b665ee6106ccaf6c4baef089ed418b70"
server_key_version: "encryptionkey@0"
I20260807 22:48:40.540850 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "8c40b7ecd39848d99a16a0f381c94a9e"
format_stamp: "Formatted at 2026-08-07 22:48:40 on dist-test-slave-csg5"
server_key: "daf2bae8f762a7c7128fe646f8d86897"
server_key_iv: "b665ee6106ccaf6c4baef089ed418b70"
server_key_version: "encryptionkey@0"
I20260807 22:48:40.547537 9315 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 22:48:40.551810 9330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:40.553397 9315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:48:40.553617 9315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "8c40b7ecd39848d99a16a0f381c94a9e"
format_stamp: "Formatted at 2026-08-07 22:48:40 on dist-test-slave-csg5"
server_key: "daf2bae8f762a7c7128fe646f8d86897"
server_key_iv: "b665ee6106ccaf6c4baef089ed418b70"
server_key_version: "encryptionkey@0"
I20260807 22:48:40.553864 9315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:40.570349 9315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:40.575939 9315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:40.576408 9315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:40.577643 9315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:40.579382 9315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:40.579504 9315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:40.579703 9315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:40.579787 9315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:40.621022 9315 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:32813
I20260807 22:48:40.621096 9450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:32813 every 8 connection(s)
I20260807 22:48:40.622885 9315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:40.628561 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9315
I20260807 22:48:40.628778 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:40.629371 20522 external_mini_cluster.cc:1428] Setting key f0d890c2dd488ded38a5cc6cd2f242bd
Aug 07 22:48:40 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142920, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.129@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:40.650122 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:40.626242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:39357 (local address 127.20.10.190:41603)
0807 22:48:40.626843 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:40.626848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:40.627943 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 22:48:40.630722 (+ 2779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:40.630727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:40.630816 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:40.630954 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:40.633223 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:40.634000 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:40.636810 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:40.637274 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:40.637421 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:40.642758 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:40.642764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:40.642774 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:40.642816 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:40.645785 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:40.647004 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:40.647009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:40.647012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:40.647153 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:40.647832 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:40.647840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:40.647846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:40.648285 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:40.648464 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:40.649295 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 22:48:40.649592 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":270,"threads_started":1}
I20260807 22:48:40.650872 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:40.626910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:41603 (local address 127.20.10.129:39357)
0807 22:48:40.627742 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:40.627792 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:40.630368 (+ 2576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:40.631269 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:40.631282 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:40.632082 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:40.632995 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:40.633024 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:40.634196 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:40.634204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:40.636575 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:40.636599 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:40.637580 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:40.637590 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:40.638025 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:40.639034 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:40.639071 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:40.642364 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:40.646001 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:40.646014 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:40.646038 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:40.646847 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:40.647277 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:40.647285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:40.647291 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:40.647601 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:40.648440 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:40.648447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:40.648859 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 22:48:40.649102 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 22:48:40.649587 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":315,"threads_started":1}
I20260807 22:48:40.652724 9451 heartbeater.cc:347] Connected to a master server at 127.20.10.190:41603
I20260807 22:48:40.653064 9451 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:40.653862 9451 heartbeater.cc:511] Master 127.20.10.190:41603 requested a full tablet report, sending...
I20260807 22:48:40.656276 9250 ts_manager.cc:194] Registered new tserver with Master: 8c40b7ecd39848d99a16a0f381c94a9e (127.20.10.129:32813)
I20260807 22:48:40.657892 9250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.129@KRBTEST.COM'} at 127.20.10.129:39357
WARNING: no policy specified for kudu/127.20.10.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
Entry for principal kudu/127.20.10.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab.
I20260807 22:48:40.716148 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:41.106010 9457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:41.106379 9457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:41.106592 9457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:41.114727 9457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:41.114854 9457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:41.115098 9457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:41.126627 9457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.130.keytab
--principal=kudu/127.20.10.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:41.128260 9457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:41.129751 9457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:41.147233 9465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:41.147154 9466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:41.147210 9468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:41.148871 9457 server_base.cc:1034] running on GCE node
I20260807 22:48:41.149642 9457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:41.150906 9457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:41.152158 9457 hybrid_clock.cc:648] HybridClock initialized: now 1786142921152088 us; error 68 us; skew 500 ppm
I20260807 22:48:41.152518 9457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:41 dist-test-slave-csg5 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142921, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:41.156291 9457 init.cc:377] Logged in from keytab as kudu/127.20.10.130@KRBTEST.COM (short username kudu)
I20260807 22:48:41.158138 9457 webserver.cc:460] Webserver started at http://127.20.10.130:37847/ using document root <none> and password file <none>
I20260807 22:48:41.159016 9457 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:41.159135 9457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:41.159441 9457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:41.162737 9457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "f8f03dd3a5d447ba9c2dfbda0657a696"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "c0ae7282f2e07309677821c878edb179"
server_key_iv: "7a457fb06d6db05ef714cb5cdb5e9f5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.163723 9457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "f8f03dd3a5d447ba9c2dfbda0657a696"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "c0ae7282f2e07309677821c878edb179"
server_key_iv: "7a457fb06d6db05ef714cb5cdb5e9f5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.170133 9457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:48:41.174271 9475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.175987 9457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:48:41.176163 9457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "f8f03dd3a5d447ba9c2dfbda0657a696"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "c0ae7282f2e07309677821c878edb179"
server_key_iv: "7a457fb06d6db05ef714cb5cdb5e9f5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.176362 9457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:41.204625 9457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:41.209645 9457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:41.209934 9457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:41.211061 9457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:41.212535 9457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:41.212615 9457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.212703 9457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:41.212749 9457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.251564 9457 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:43781
I20260807 22:48:41.251657 9604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:43781 every 8 connection(s)
I20260807 22:48:41.253098 9457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:41.256052 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9457
I20260807 22:48:41.256268 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:41.256891 20522 external_mini_cluster.cc:1428] Setting key ea8458a8d8ca59234d520be252c79b53
Aug 07 22:48:41 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142921, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.130@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:41.281672 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.256105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:49135 (local address 127.20.10.190:41603)
0807 22:48:41.256740 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.256747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.257902 (+ 1155us) server_negotiation.cc:407] Connection header received
0807 22:48:41.260381 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.260390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.260502 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.260624 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:41.262370 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.263645 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.265430 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.266201 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.266353 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.274631 (+ 8278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:41.274637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:41.274640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:41.274736 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:41.277773 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.278741 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.278747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.278749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.278893 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.279397 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.279403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.279406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.279716 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:41.279890 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.280837 (+ 947us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.281167 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":365,"threads_started":1}
I20260807 22:48:41.282822 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.256725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:41603 (local address 127.20.10.130:49135)
0807 22:48:41.257671 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:41.257737 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:41.260062 (+ 2325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:41.260917 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:41.260926 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:41.261540 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:41.262174 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.262192 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.263867 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.263871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.265252 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.265266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.266478 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.266484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.266798 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.271458 (+ 4660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:41.271490 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:41.274296 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:41.277987 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:41.277998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:41.278019 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:41.278542 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:41.279028 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:41.279034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:41.279037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:41.279265 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:41.279888 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:41.279898 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:41.280419 (+ 521us) client_negotiation.cc:769] Sending connection context
0807 22:48:41.280745 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 22:48:41.281379 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":415,"threads_started":1}
I20260807 22:48:41.284785 9605 heartbeater.cc:347] Connected to a master server at 127.20.10.190:41603
I20260807 22:48:41.285192 9605 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:41.286096 9605 heartbeater.cc:511] Master 127.20.10.190:41603 requested a full tablet report, sending...
I20260807 22:48:41.288038 9250 ts_manager.cc:194] Registered new tserver with Master: f8f03dd3a5d447ba9c2dfbda0657a696 (127.20.10.130:43781)
I20260807 22:48:41.289446 9250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.130@KRBTEST.COM'} at 127.20.10.130:49135
WARNING: no policy specified for kudu/127.20.10.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
Entry for principal kudu/127.20.10.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab.
I20260807 22:48:41.343115 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:48:41.661374 9451 heartbeater.cc:503] Master 127.20.10.190:41603 was elected leader, sending a full tablet report...
W20260807 22:48:41.729745 9612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:41.730137 9612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:41.730273 9612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:41.738067 9612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:41.738204 9612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:41.738407 9612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:41.749092 9612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.131.keytab
--principal=kudu/127.20.10.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:41.750579 9612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:41.752027 9612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:41.766984 9617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:41.767094 9621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:41.767094 9618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:41.768630 9612 server_base.cc:1034] running on GCE node
I20260807 22:48:41.769481 9612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:41.770550 9612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:41.771833 9612 hybrid_clock.cc:648] HybridClock initialized: now 1786142921771793 us; error 64 us; skew 500 ppm
I20260807 22:48:41.772218 9612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:41 dist-test-slave-csg5 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142921, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:41.775940 9612 init.cc:377] Logged in from keytab as kudu/127.20.10.131@KRBTEST.COM (short username kudu)
I20260807 22:48:41.777702 9612 webserver.cc:460] Webserver started at http://127.20.10.131:32817/ using document root <none> and password file <none>
I20260807 22:48:41.778580 9612 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:41.778780 9612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:41.779129 9612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:41.782913 9612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "3502af08b14248368a9ab18b5e494728"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "bde780692c5274e87ef28c0417ca4da4"
server_key_iv: "2e2cd2f731e926f20f9c84a2780756c9"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.783996 9612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "3502af08b14248368a9ab18b5e494728"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "bde780692c5274e87ef28c0417ca4da4"
server_key_iv: "2e2cd2f731e926f20f9c84a2780756c9"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.790028 9612 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 22:48:41.794095 9628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.795819 9612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:48:41.796033 9612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "3502af08b14248368a9ab18b5e494728"
format_stamp: "Formatted at 2026-08-07 22:48:41 on dist-test-slave-csg5"
server_key: "bde780692c5274e87ef28c0417ca4da4"
server_key_iv: "2e2cd2f731e926f20f9c84a2780756c9"
server_key_version: "encryptionkey@0"
I20260807 22:48:41.796275 9612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:41.826052 9612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:41.831089 9612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:41.831418 9612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:41.832543 9612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:41.834048 9612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:41.834162 9612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.834308 9612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:41.834384 9612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:41.872743 9612 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:34775
I20260807 22:48:41.872844 9755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:34775 every 8 connection(s)
I20260807 22:48:41.874111 9612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:41.879182 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9612
I20260807 22:48:41.879402 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:41.880030 20522 external_mini_cluster.cc:1428] Setting key 97cdaa4306785ec254d8a62e3de0678e
Aug 07 22:48:41 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142921, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.131@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:48:41.902488 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.877326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:60165 (local address 127.20.10.190:41603)
0807 22:48:41.877843 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.877850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.879026 (+ 1176us) server_negotiation.cc:407] Connection header received
0807 22:48:41.881825 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.881830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.881960 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.882099 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:41.884792 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.885760 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.887742 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.888305 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.888434 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.893001 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:41.893008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:41.893012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:41.893073 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:41.898159 (+ 5086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.899495 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.899500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.899502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.899684 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.900257 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.900264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.900267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.900664 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:41.900853 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.901915 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.902138 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":233,"threads_started":1}
I20260807 22:48:41.903666 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.877854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:41603 (local address 127.20.10.131:60165)
0807 22:48:41.878798 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:41.878865 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:41.881441 (+ 2576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:41.882532 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:41.882543 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:41.883704 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:41.884480 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.884503 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.886006 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.886010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.887541 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.887557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.888576 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.888582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.888878 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.889816 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:48:41.889843 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:41.892604 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:41.898429 (+ 5825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:41.898443 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:41.898474 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:41.899314 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:41.899855 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:48:41.899859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:48:41.899862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:48:41.900111 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:48:41.900821 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:41.900828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:41.901338 (+ 510us) client_negotiation.cc:769] Sending connection context
0807 22:48:41.901696 (+ 358us) client_negotiation.cc:240] Negotiation successful
0807 22:48:41.902177 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":374,"threads_started":1}
I20260807 22:48:41.905542 9757 heartbeater.cc:347] Connected to a master server at 127.20.10.190:41603
I20260807 22:48:41.905934 9757 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:41.906885 9757 heartbeater.cc:511] Master 127.20.10.190:41603 requested a full tablet report, sending...
I20260807 22:48:41.908870 9250 ts_manager.cc:194] Registered new tserver with Master: 3502af08b14248368a9ab18b5e494728 (127.20.10.131:34775)
I20260807 22:48:41.910041 9250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.10.131@KRBTEST.COM'} at 127.20.10.131:60165
I20260807 22:48:41.911501 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:41.931926 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.916446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.20.10.190:41603)
0807 22:48:41.916703 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.916710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.917251 (+ 541us) server_negotiation.cc:407] Connection header received
0807 22:48:41.917415 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.917419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.917508 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.917610 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:41.919441 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.920347 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.921940 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.922365 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.922476 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.925296 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:41.925301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:41.925305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:41.925353 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:41.928761 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.929632 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.929638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.929641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.929788 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:41.930159 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:41.930163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:41.930165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:41.930539 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:41.930866 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.931307 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.931568 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:48:41.936748 9250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:41.945428 9250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:41.985865 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.961551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.20.10.130:43781)
0807 22:48:41.976164 (+ 14613us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.976186 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.976220 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:48:41.976329 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.976337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.976553 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.977300 (+ 747us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:41.978336 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.979983 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.982974 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984438 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.984917 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.985160 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.985319 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.985504 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14423,"thread_start_us":14416,"threads_started":1}
I20260807 22:48:41.985885 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.960844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:43781 (local address 127.0.0.1:41916)
0807 22:48:41.969235 (+ 8391us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:41.969320 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:41.969664 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:41.976811 (+ 7147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:41.976814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:41.976831 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:41.977824 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.977845 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.980151 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.980161 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.982780 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.982796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984739 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.984748 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.985111 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.985167 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:48:41.985265 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:48:41.985386 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":410,"thread_start_us":532,"threads_started":1}
I20260807 22:48:41.985951 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.961340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:34775 (local address 127.0.0.1:36012)
0807 22:48:41.975545 (+ 14205us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:41.975569 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:41.975741 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:41.977115 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:41.977119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:41.977150 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:41.977856 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.977868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.979770 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.979777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.982584 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.982607 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984778 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.984784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.985122 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.985177 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 22:48:41.985263 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:48:41.985415 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14030,"mutex_wait_us":156,"spinlock_wait_cycles":7296,"thread_start_us":13775,"threads_started":1}
I20260807 22:48:41.986734 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.962421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:32813 (local address 127.0.0.1:38054)
0807 22:48:41.975548 (+ 13127us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:41.975569 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:41.975741 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:41.976769 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:41.976781 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:41.976820 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:41.977524 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.977550 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.979796 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.979801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.982578 (+ 2777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:41.982601 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984470 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.984476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:41.984829 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.984911 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 22:48:41.985077 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 22:48:41.985319 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12999,"mutex_wait_us":142,"spinlock_wait_cycles":12928,"thread_start_us":13408,"threads_started":1}
I20260807 22:48:41.987260 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.962236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36012 (local address 127.20.10.131:34775)
0807 22:48:41.976521 (+ 14285us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.976529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.976566 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:48:41.976664 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.976671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.976886 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.977110 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:41.978082 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.979623 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.982852 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984438 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.986368 (+ 1930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.986635 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.986728 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.986901 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14097,"thread_start_us":13960,"threads_started":1}
I20260807 22:48:41.987629 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:41.962993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.20.10.129:32813)
0807 22:48:41.973137 (+ 10144us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:41.973150 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:41.975906 (+ 2756us) server_negotiation.cc:407] Connection header received
0807 22:48:41.976045 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:41.976058 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:41.976438 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:41.976694 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:41.977801 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.979627 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.982844 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:41.984244 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:41.984420 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:41.984707 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:41.985698 (+ 991us) server_negotiation.cc:299] Negotiation successful
0807 22:48:41.985908 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9893,"thread_start_us":9679,"threads_started":1}
I20260807 22:48:41.990563 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4435dc0ff45528a72b8658e85c81e (DEFAULT_TABLE table=test-table [id=126ef003c3d249f985e37ad7ddcac992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:41.992081 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4435dc0ff45528a72b8658e85c81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:41.992084 9682 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4435dc0ff45528a72b8658e85c81e (DEFAULT_TABLE table=test-table [id=126ef003c3d249f985e37ad7ddcac992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:41.992086 9382 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4435dc0ff45528a72b8658e85c81e (DEFAULT_TABLE table=test-table [id=126ef003c3d249f985e37ad7ddcac992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:41.993887 9682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4435dc0ff45528a72b8658e85c81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:41.994277 9382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4435dc0ff45528a72b8658e85c81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:42.004637 9779 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Bootstrap starting.
I20260807 22:48:42.009011 9779 tablet_bootstrap.cc:654] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:42.010156 9779 log.cc:824] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:42.011853 9781 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Bootstrap starting.
I20260807 22:48:42.013957 9779 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: No bootstrap required, opened a new log
I20260807 22:48:42.014256 9779 ts_tablet_manager.cc:1397] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260807 22:48:42.015573 9782 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Bootstrap starting.
I20260807 22:48:42.015861 9781 tablet_bootstrap.cc:654] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:42.017704 9781 log.cc:824] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:42.019658 9782 tablet_bootstrap.cc:654] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:42.021162 9782 log.cc:824] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:42.021938 9781 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: No bootstrap required, opened a new log
I20260807 22:48:42.022233 9781 ts_tablet_manager.cc:1397] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260807 22:48:42.024458 9782 tablet_bootstrap.cc:492] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: No bootstrap required, opened a new log
I20260807 22:48:42.024785 9782 ts_tablet_manager.cc:1397] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:48:42.027099 9779 raft_consensus.cc:348] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.028169 9779 raft_consensus.cc:374] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:42.028276 9779 raft_consensus.cc:729] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f03dd3a5d447ba9c2dfbda0657a696, State: Initialized, Role: FOLLOWER
I20260807 22:48:42.029101 9779 consensus_queue.cc:260] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.030424 9605 heartbeater.cc:503] Master 127.20.10.190:41603 was elected leader, sending a full tablet report...
I20260807 22:48:42.031195 9779 ts_tablet_manager.cc:1428] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Time spent starting tablet: real 0.017s user 0.007s sys 0.009s
I20260807 22:48:42.034019 9781 raft_consensus.cc:348] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.034443 9781 raft_consensus.cc:374] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:42.034581 9781 raft_consensus.cc:729] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3502af08b14248368a9ab18b5e494728, State: Initialized, Role: FOLLOWER
I20260807 22:48:42.035091 9782 raft_consensus.cc:348] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.035369 9781 consensus_queue.cc:260] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.035625 9782 raft_consensus.cc:374] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:42.035706 9782 raft_consensus.cc:729] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c40b7ecd39848d99a16a0f381c94a9e, State: Initialized, Role: FOLLOWER
I20260807 22:48:42.036496 9782 consensus_queue.cc:260] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.036712 9757 heartbeater.cc:503] Master 127.20.10.190:41603 was elected leader, sending a full tablet report...
I20260807 22:48:42.037504 9781 ts_tablet_manager.cc:1428] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 22:48:42.039415 9782 ts_tablet_manager.cc:1428] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Time spent starting tablet: real 0.014s user 0.010s sys 0.006s
W20260807 22:48:42.126025 9758 tablet.cc:2369] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:48:42.126127 9452 tablet.cc:2369] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:42.157713 9787 raft_consensus.cc:482] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:42.158102 9787 raft_consensus.cc:504] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.160115 9787 leader_election.cc:288] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c40b7ecd39848d99a16a0f381c94a9e (127.20.10.129:32813), 3502af08b14248368a9ab18b5e494728 (127.20.10.131:34775)
I20260807 22:48:42.167695 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.160603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:38805 (local address 127.20.10.131:34775)
0807 22:48:42.161059 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.161068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.162024 (+ 956us) server_negotiation.cc:407] Connection header received
0807 22:48:42.162293 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.162298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.162365 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.162470 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:42.163139 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.163922 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.165809 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.166645 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.166906 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.166986 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.167265 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.167399 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260807 22:48:42.167784 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.160491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:34775 (local address 127.20.10.130:38805)
0807 22:48:42.161869 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:42.161890 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:42.162128 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:42.162493 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:42.162497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:42.162531 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:42.162989 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:42.162999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.164062 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.164068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:42.165639 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:42.165652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.166924 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.166931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:42.167096 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.167121 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:48:42.167236 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:48:42.167400 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":448,"threads_started":1}
I20260807 22:48:42.168176 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.160640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:32813 (local address 127.20.10.130:38779)
0807 22:48:42.162370 (+ 1730us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:42.162387 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:42.162781 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:42.163133 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:42.163138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:42.163162 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:42.163561 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:42.163571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.164623 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.164638 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:42.166066 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:42.166077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.167353 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.167360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:42.167547 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.167569 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:48:42.167685 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 22:48:42.167831 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1515,"thread_start_us":1260,"threads_started":1}
I20260807 22:48:42.168766 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.160584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:38779 (local address 127.20.10.129:32813)
0807 22:48:42.160827 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.160834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.162569 (+ 1735us) server_negotiation.cc:407] Connection header received
0807 22:48:42.162945 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.162956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.163019 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.163120 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:42.163719 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.164479 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.166232 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.167111 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.167323 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.167403 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.167714 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.167845 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:48:42.169363 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4435dc0ff45528a72b8658e85c81e" candidate_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3502af08b14248368a9ab18b5e494728" is_pre_election: true
I20260807 22:48:42.170022 9704 raft_consensus.cc:2454] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f03dd3a5d447ba9c2dfbda0657a696 in term 0.
I20260807 22:48:42.170113 9403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4435dc0ff45528a72b8658e85c81e" candidate_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" is_pre_election: true
I20260807 22:48:42.170770 9403 raft_consensus.cc:2454] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f03dd3a5d447ba9c2dfbda0657a696 in term 0.
I20260807 22:48:42.170938 9476 leader_election.cc:302] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3502af08b14248368a9ab18b5e494728, f8f03dd3a5d447ba9c2dfbda0657a696; no voters:
I20260807 22:48:42.171445 9787 raft_consensus.cc:2781] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:42.171624 9787 raft_consensus.cc:482] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:42.171737 9787 raft_consensus.cc:3037] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:42.174162 9787 raft_consensus.cc:504] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.174944 9787 leader_election.cc:288] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [CANDIDATE]: Term 1 election: Requested vote from peers 8c40b7ecd39848d99a16a0f381c94a9e (127.20.10.129:32813), 3502af08b14248368a9ab18b5e494728 (127.20.10.131:34775)
I20260807 22:48:42.175804 9403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4435dc0ff45528a72b8658e85c81e" candidate_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c40b7ecd39848d99a16a0f381c94a9e"
I20260807 22:48:42.175786 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4435dc0ff45528a72b8658e85c81e" candidate_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3502af08b14248368a9ab18b5e494728"
I20260807 22:48:42.176077 9403 raft_consensus.cc:3037] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:42.176088 9704 raft_consensus.cc:3037] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:42.180140 9704 raft_consensus.cc:2454] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f03dd3a5d447ba9c2dfbda0657a696 in term 1.
I20260807 22:48:42.180112 9403 raft_consensus.cc:2454] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f03dd3a5d447ba9c2dfbda0657a696 in term 1.
I20260807 22:48:42.180814 9476 leader_election.cc:302] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3502af08b14248368a9ab18b5e494728, f8f03dd3a5d447ba9c2dfbda0657a696; no voters:
I20260807 22:48:42.181303 9787 raft_consensus.cc:2781] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:42.181955 9787 raft_consensus.cc:686] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 1 LEADER]: Becoming Leader. State: Replica: f8f03dd3a5d447ba9c2dfbda0657a696, State: Running, Role: LEADER
I20260807 22:48:42.182546 9787 consensus_queue.cc:237] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } }
I20260807 22:48:42.188306 9250 catalog_manager.cc:5495] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8f03dd3a5d447ba9c2dfbda0657a696 (127.20.10.130). New cstate: current_term: 1 leader_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f03dd3a5d447ba9c2dfbda0657a696" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 } health_report { overall_health: UNKNOWN } } }
I20260807 22:48:42.233451 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.226553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.20.10.130:43781)
0807 22:48:42.226930 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.226939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.227062 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:48:42.227381 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.227385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.227493 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.227712 (+ 219us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:42.228174 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.229055 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.230834 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.231312 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.231470 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.232119 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:42.232725 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:42.232912 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.233014 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.233123 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260807 22:48:42.243251 9704 raft_consensus.cc:1261] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 1 FOLLOWER]: Refusing update from remote peer f8f03dd3a5d447ba9c2dfbda0657a696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:42.243440 9403 raft_consensus.cc:1261] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 1 FOLLOWER]: Refusing update from remote peer f8f03dd3a5d447ba9c2dfbda0657a696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:42.244628 9792 consensus_queue.cc:1035] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3502af08b14248368a9ab18b5e494728" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:42.245322 9787 consensus_queue.cc:1035] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c40b7ecd39848d99a16a0f381c94a9e" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 32813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:48:42.256986 9606 tablet.cc:2369] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:42.262181 9795 mvcc.cc:205] Tried to move back new op lower bound from 7316041409496117248 to 7316041409264955392. Current Snapshot: MvccSnapshot[applied={T|T < 7316041409496117248}]
I20260807 22:48:42.265722 9796 mvcc.cc:205] Tried to move back new op lower bound from 7316041409496117248 to 7316041409264955392. Current Snapshot: MvccSnapshot[applied={T|T < 7316041409496117248}]
I20260807 22:48:42.289708 9249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:42.290059 9249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:42.294446 9249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56ca11ce7f874e079e8307a7bab46339: Sending DeleteTablet for 3 replicas of tablet 72a4435dc0ff45528a72b8658e85c81e
I20260807 22:48:42.296026 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4435dc0ff45528a72b8658e85c81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:41916
I20260807 22:48:42.296031 9682 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4435dc0ff45528a72b8658e85c81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:36012
I20260807 22:48:42.297109 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9315
I20260807 22:48:42.296989 9382 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4435dc0ff45528a72b8658e85c81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:42 UTC) from {username='kudu', principal='kudu/127.20.10.190@KRBTEST.COM'} at 127.0.0.1:38054
I20260807 22:48:42.302947 9806 tablet_replica.cc:330] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: stopping tablet replica
I20260807 22:48:42.303601 9806 raft_consensus.cc:2229] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:42.304348 9806 raft_consensus.cc:2258] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:42.307824 9806 ts_tablet_manager.cc:1905] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 22:48:42 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.129@KRBTEST.COM
I20260807 22:48:42.319742 9805 tablet_replica.cc:330] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: stopping tablet replica
I20260807 22:48:42.320324 9805 raft_consensus.cc:2229] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:42.320972 9804 tablet_replica.cc:330] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: stopping tablet replica
I20260807 22:48:42.321458 9804 raft_consensus.cc:2229] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:42.322134 9804 raft_consensus.cc:2258] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:42.322175 9805 raft_consensus.cc:2258] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:42.324697 9804 ts_tablet_manager.cc:1905] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:42.326855 9806 ts_tablet_manager.cc:1918] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:42.327169 9806 log.cc:1196] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/72a4435dc0ff45528a72b8658e85c81e
I20260807 22:48:42.326606 9805 ts_tablet_manager.cc:1905] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:42.328006 9806 ts_tablet_manager.cc:1939] T 72a4435dc0ff45528a72b8658e85c81e P 8c40b7ecd39848d99a16a0f381c94a9e: Deleting consensus metadata
I20260807 22:48:42.330426 9236 catalog_manager.cc:4841] TS 8c40b7ecd39848d99a16a0f381c94a9e (127.20.10.129:32813): tablet 72a4435dc0ff45528a72b8658e85c81e (table test-table [id=126ef003c3d249f985e37ad7ddcac992]) successfully deleted
I20260807 22:48:42.332295 9804 ts_tablet_manager.cc:1918] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:42.332479 9804 log.cc:1196] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/72a4435dc0ff45528a72b8658e85c81e
I20260807 22:48:42.333068 9804 ts_tablet_manager.cc:1939] T 72a4435dc0ff45528a72b8658e85c81e P 3502af08b14248368a9ab18b5e494728: Deleting consensus metadata
I20260807 22:48:42.334739 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.299233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.20.10.129:32813)
0807 22:48:42.299487 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.299496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.302316 (+ 2820us) server_negotiation.cc:407] Connection header received
0807 22:48:42.302579 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.302588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.302717 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.302827 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:42.304928 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.306542 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.309370 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.310427 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.310585 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.318805 (+ 8220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:42.318830 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:42.318860 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:42.318940 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:42.324256 (+ 5316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.331009 (+ 6753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.331021 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.331024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.331269 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.331927 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.331933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.331937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.332554 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:42.332908 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.333905 (+ 997us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.334259 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:48:42.335083 9233 catalog_manager.cc:4841] TS 3502af08b14248368a9ab18b5e494728 (127.20.10.131:34775): tablet 72a4435dc0ff45528a72b8658e85c81e (table test-table [id=126ef003c3d249f985e37ad7ddcac992]) successfully deleted
I20260807 22:48:42.336880 9446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:42.337602 9805 ts_tablet_manager.cc:1918] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:42.337850 9805 log.cc:1196] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/72a4435dc0ff45528a72b8658e85c81e
I20260807 22:48:42.338446 9805 ts_tablet_manager.cc:1939] T 72a4435dc0ff45528a72b8658e85c81e P f8f03dd3a5d447ba9c2dfbda0657a696: Deleting consensus metadata
I20260807 22:48:42.341704 9236 catalog_manager.cc:4841] TS f8f03dd3a5d447ba9c2dfbda0657a696 (127.20.10.130:43781): tablet 72a4435dc0ff45528a72b8658e85c81e (table test-table [id=126ef003c3d249f985e37ad7ddcac992]) successfully deleted
I20260807 22:48:42.445434 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9315
I20260807 22:48:42.469956 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9457
Aug 07 22:48:42 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.130@KRBTEST.COM
I20260807 22:48:42.490137 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.470939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.20.10.130:43781)
0807 22:48:42.471125 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.471130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.471150 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:48:42.471331 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.471335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.471423 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.471521 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:42.473361 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.474194 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.476396 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.476887 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.476996 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.483227 (+ 6231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:42.483252 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:42.483269 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:42.483329 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:42.486863 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.487760 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.487769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.487771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.487962 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.488406 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.488410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.488412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.488799 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:42.489169 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.489482 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.489747 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:48:42.490918 9600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:42.617300 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9457
I20260807 22:48:42.643247 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9612
Aug 07 22:48:42 dist-test-slave-csg5 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.131@KRBTEST.COM
I20260807 22:48:42.661334 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:42.644180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.20.10.131:34775)
0807 22:48:42.644506 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:42.644511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:42.644531 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:48:42.644591 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:42.644594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:42.644682 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:42.644781 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:42.646326 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.647212 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.648632 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:42.649027 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:42.649134 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:42.654302 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:42.654322 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:42.654343 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:42.654388 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:42.657709 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.658561 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.658569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.658572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.658911 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:42.659438 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:42.659442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:42.659444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:42.659954 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:42.660198 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:42.660682 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 22:48:42.660907 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260807 22:48:42.662098 9751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:42.757645 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9612
I20260807 22:48:42.780697 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9217
I20260807 22:48:42.782159 9284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:42.900184 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9217
2026-08-07T22:48:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5166 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T22:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:42Z Disabled control of system clock
I20260807 22:48:43.012193 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33641
--webserver_interface=127.20.10.190
--webserver_port=0
--builtin_ntp_servers=127.20.10.148:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:43.387856 9815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:43.388314 9815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:43.388558 9815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:43.397680 9815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:43.397833 9815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:43.397876 9815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:43.397930 9815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:43.397979 9815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:43.408183 9815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:43.409643 9815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:43.411773 9815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:43.425755 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:43.425710 9820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:43.425838 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:43.427707 9815 server_base.cc:1034] running on GCE node
I20260807 22:48:43.428753 9815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:43.430524 9815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:43.431856 9815 hybrid_clock.cc:648] HybridClock initialized: now 1786142923431821 us; error 77 us; skew 500 ppm
I20260807 22:48:43.432375 9815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:43.435678 9815 webserver.cc:460] Webserver started at http://127.20.10.190:42255/ using document root <none> and password file <none>
I20260807 22:48:43.436501 9815 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:43.436657 9815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:43.437021 9815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:43.440435 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "83935a64cc874b5291eccfffc3b18424"
format_stamp: "Formatted at 2026-08-07 22:48:43 on dist-test-slave-csg5"
server_key: "102a79bd5ac32ddbc1f8d37174cc8581"
server_key_iv: "c3bea879ff3b6ab1222cffab250f1d5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:43.441625 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "83935a64cc874b5291eccfffc3b18424"
format_stamp: "Formatted at 2026-08-07 22:48:43 on dist-test-slave-csg5"
server_key: "102a79bd5ac32ddbc1f8d37174cc8581"
server_key_iv: "c3bea879ff3b6ab1222cffab250f1d5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:43.448072 9815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 22:48:43.452359 9830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:43.454181 9815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:48:43.454365 9815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "83935a64cc874b5291eccfffc3b18424"
format_stamp: "Formatted at 2026-08-07 22:48:43 on dist-test-slave-csg5"
server_key: "102a79bd5ac32ddbc1f8d37174cc8581"
server_key_iv: "c3bea879ff3b6ab1222cffab250f1d5d"
server_key_version: "encryptionkey@0"
I20260807 22:48:43.454725 9815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:43.478916 9815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:43.479979 9815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:43.480368 9815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:43.503687 9815 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33641
I20260807 22:48:43.503736 9887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33641 every 8 connection(s)
I20260807 22:48:43.505210 9815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:43.509719 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:43.511864 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9815
I20260807 22:48:43.512079 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:43.512773 20522 external_mini_cluster.cc:1428] Setting key 3a00539770e907f1ebd2f95b5ee6afab
I20260807 22:48:43.525183 9888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: Bootstrap starting.
I20260807 22:48:43.528383 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:43.515233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.20.10.190:33641)
0807 22:48:43.516547 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:43.516560 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:43.516608 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:48:43.518430 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:43.518458 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:43.519118 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:43.519578 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:43.520208 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:43.521556 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:43.523063 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:43.523789 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:43.524035 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:43.525440 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:43.525461 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:43.525478 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:43.526083 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:43.526173 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:43.526198 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:43.526361 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:43.526610 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 22:48:43.527008 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":544,"threads_started":1}
I20260807 22:48:43.529738 9888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:43.531085 9888 log.cc:824] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:43.534924 9888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: No bootstrap required, opened a new log
I20260807 22:48:43.543037 9888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } }
I20260807 22:48:43.543516 9888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:43.543633 9888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83935a64cc874b5291eccfffc3b18424, State: Initialized, Role: FOLLOWER
I20260807 22:48:43.544270 9888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } }
I20260807 22:48:43.544539 9888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:43.544705 9888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:43.544889 9888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:43.547410 9888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } }
I20260807 22:48:43.548046 9888 leader_election.cc:302] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83935a64cc874b5291eccfffc3b18424; no voters:
I20260807 22:48:43.548811 9888 leader_election.cc:288] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:43.549053 9896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:43.549916 9896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [term 1 LEADER]: Becoming Leader. State: Replica: 83935a64cc874b5291eccfffc3b18424, State: Running, Role: LEADER
I20260807 22:48:43.550462 9896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } }
I20260807 22:48:43.551173 9888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:43.553793 9897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83935a64cc874b5291eccfffc3b18424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } } }
I20260807 22:48:43.553629 9898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83935a64cc874b5291eccfffc3b18424. Latest consensus state: current_term: 1 leader_uuid: "83935a64cc874b5291eccfffc3b18424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83935a64cc874b5291eccfffc3b18424" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33641 } } }
I20260807 22:48:43.554251 9897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:43.554250 9898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:43.557325 9904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:43.562152 9904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:43.571100 9904 catalog_manager.cc:1347] Generated new cluster ID: eca82b7cf6ad49d7accf3c227ed6e1b8
I20260807 22:48:43.571244 9904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:43.594058 9904 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:43.595574 9904 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:43.604374 9904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: Generated new TSK 0
I20260807 22:48:43.605320 9904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:43.612170 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.129:0
--local_ip_for_outbound_sockets=127.20.10.129
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--builtin_ntp_servers=127.20.10.148:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:48:43.999742 9915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:44.000088 9915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:44.000195 9915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:44.009169 9915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:44.009292 9915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:44.009488 9915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.129
I20260807 22:48:44.020699 9915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.10.129
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.129
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:44.022171 9915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:44.024312 9915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:44.040014 9922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:44.040117 9921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:44.040033 9924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:44.042219 9915 server_base.cc:1034] running on GCE node
I20260807 22:48:44.043136 9915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:44.044414 9915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:44.045720 9915 hybrid_clock.cc:648] HybridClock initialized: now 1786142924045660 us; error 48 us; skew 500 ppm
I20260807 22:48:44.046159 9915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:44.049203 9915 webserver.cc:460] Webserver started at http://127.20.10.129:42001/ using document root <none> and password file <none>
I20260807 22:48:44.050009 9915 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:44.050155 9915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:44.050516 9915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:44.054399 9915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data/instance:
uuid: "1f33391fc79147caa42a2697197ba668"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "44e93c0da2629d4f6c8c867542030244"
server_key_iv: "6a6e56424d37252428173bb1ed2c922f"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.055447 9915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance:
uuid: "1f33391fc79147caa42a2697197ba668"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "44e93c0da2629d4f6c8c867542030244"
server_key_iv: "6a6e56424d37252428173bb1ed2c922f"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.061620 9915 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:48:44.066264 9931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:44.067989 9915 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:48:44.068249 9915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal
uuid: "1f33391fc79147caa42a2697197ba668"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "44e93c0da2629d4f6c8c867542030244"
server_key_iv: "6a6e56424d37252428173bb1ed2c922f"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.068570 9915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:44.095191 9915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:44.096215 9915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:44.096585 9915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:44.098067 9915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:44.099928 9915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:44.100025 9915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:44.100200 9915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:44.100317 9915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:44.141093 9915 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.129:36455
I20260807 22:48:44.141225 10052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.129:36455 every 8 connection(s)
I20260807 22:48:44.142733 9915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/data/info.pb
I20260807 22:48:44.151839 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 9915
I20260807 22:48:44.152062 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal/instance
I20260807 22:48:44.152665 20522 external_mini_cluster.cc:1428] Setting key 6ec316278848b76546a6ac5f6829286e
I20260807 22:48:44.156275 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.130:0
--local_ip_for_outbound_sockets=127.20.10.130
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--builtin_ntp_servers=127.20.10.148:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:44.159015 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:44.145987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:37941 (local address 127.20.10.190:33641)
0807 22:48:44.146552 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:44.146557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:44.148059 (+ 1502us) server_negotiation.cc:407] Connection header received
0807 22:48:44.150144 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:44.150150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:44.150253 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:44.150381 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:44.151858 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.153032 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.155206 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.155752 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.155872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:44.157805 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:44.157814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:44.157820 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:44.157858 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:44.157903 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:44.157919 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:44.158047 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:44.158401 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 22:48:44.158512 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260807 22:48:44.160444 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:44.146828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33641 (local address 127.20.10.129:37941)
0807 22:48:44.147876 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:44.147922 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:44.149839 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:44.150650 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:44.150709 (+ 59us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:44.150829 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:44.151635 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:44.151665 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.153230 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.153236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:44.155004 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:44.155019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.156041 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.156053 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:44.156416 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:44.157317 (+ 901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:44.157324 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:44.157391 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:44.157523 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:44.158052 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:44.158068 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:44.158081 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:48:44.158316 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 22:48:44.158868 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":352,"threads_started":1}
I20260807 22:48:44.162313 10053 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33641
I20260807 22:48:44.162815 10053 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:44.163774 10053 heartbeater.cc:511] Master 127.20.10.190:33641 requested a full tablet report, sending...
I20260807 22:48:44.166330 9850 ts_manager.cc:194] Registered new tserver with Master: 1f33391fc79147caa42a2697197ba668 (127.20.10.129:36455)
I20260807 22:48:44.168078 9850 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.129:37941
W20260807 22:48:44.547189 10057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:44.547577 10057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:44.547760 10057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:44.556833 10057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:44.556994 10057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:44.557200 10057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.130
I20260807 22:48:44.568609 10057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.10.130
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.130
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:44.570166 10057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:44.572562 10057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:44.588712 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:44.588828 10063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:44.588828 10064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:44.589815 10057 server_base.cc:1034] running on GCE node
I20260807 22:48:44.590471 10057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:44.591681 10057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:44.592916 10057 hybrid_clock.cc:648] HybridClock initialized: now 1786142924592862 us; error 45 us; skew 500 ppm
I20260807 22:48:44.593283 10057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:44.595995 10057 webserver.cc:460] Webserver started at http://127.20.10.130:40703/ using document root <none> and password file <none>
I20260807 22:48:44.596776 10057 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:44.596903 10057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:44.597281 10057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:44.600574 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data/instance:
uuid: "13069874cf014e5f9c8450f1f5aed42c"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "24245643010af4662d47e31d909524e0"
server_key_iv: "340e2eb71caa5314757af15b42484948"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.601456 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance:
uuid: "13069874cf014e5f9c8450f1f5aed42c"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "24245643010af4662d47e31d909524e0"
server_key_iv: "340e2eb71caa5314757af15b42484948"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.607506 10057 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:48:44.611405 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:44.612951 10057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:48:44.613155 10057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal
uuid: "13069874cf014e5f9c8450f1f5aed42c"
format_stamp: "Formatted at 2026-08-07 22:48:44 on dist-test-slave-csg5"
server_key: "24245643010af4662d47e31d909524e0"
server_key_iv: "340e2eb71caa5314757af15b42484948"
server_key_version: "encryptionkey@0"
I20260807 22:48:44.613484 10057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:44.649734 10057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:44.650842 10057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:44.651197 10057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:44.652535 10057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:44.654325 10057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:44.654470 10057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:44.654620 10057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:44.654734 10057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 22:48:44.694157 10057 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.130:37449
I20260807 22:48:44.694260 10194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.130:37449 every 8 connection(s)
I20260807 22:48:44.695683 10057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/data/info.pb
I20260807 22:48:44.697219 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10057
I20260807 22:48:44.697410 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal/instance
I20260807 22:48:44.698031 20522 external_mini_cluster.cc:1428] Setting key 0e0e7c692b20de4c076dc937babf0eca
I20260807 22:48:44.701797 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.10.131:0
--local_ip_for_outbound_sockets=127.20.10.131
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--builtin_ntp_servers=127.20.10.148:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:48:44.713621 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:44.699135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.130:35151 (local address 127.20.10.190:33641)
0807 22:48:44.699710 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:44.699717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:44.701565 (+ 1848us) server_negotiation.cc:407] Connection header received
0807 22:48:44.704874 (+ 3309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:44.704879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:44.704973 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:44.705090 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:44.706832 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.707641 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.710281 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.710799 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.710914 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:44.712749 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:44.712754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:44.712757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:44.712780 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:44.712833 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:44.712846 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:44.712936 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:44.713227 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 22:48:44.713330 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":283,"threads_started":1}
I20260807 22:48:44.715292 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:44.699933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33641 (local address 127.20.10.130:35151)
0807 22:48:44.701371 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:44.701431 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:44.704453 (+ 3022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:44.705452 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:44.705467 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:44.705612 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:44.706564 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:44.706595 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.707821 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.707829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:44.710079 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:44.710100 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:44.711061 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:44.711071 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:44.711449 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:44.712304 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:44.712310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:44.712379 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:44.712509 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:44.712967 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:44.712981 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:44.712992 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:48:44.713208 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 22:48:44.713683 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":444,"threads_started":1}
I20260807 22:48:44.717096 10195 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33641
I20260807 22:48:44.717558 10195 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:44.718587 10195 heartbeater.cc:511] Master 127.20.10.190:33641 requested a full tablet report, sending...
I20260807 22:48:44.720445 9850 ts_manager.cc:194] Registered new tserver with Master: 13069874cf014e5f9c8450f1f5aed42c (127.20.10.130:37449)
I20260807 22:48:44.721329 9850 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.130:35151
W20260807 22:48:45.083384 10200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:45.083767 10200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:45.083946 10200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:45.091626 10200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:45.091804 10200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:45.091989 10200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.10.131
I20260807 22:48:45.101826 10200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.10.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.10.131
--webserver_port=0
--tserver_master_addrs=127.20.10.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.10.131
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:45.103432 10200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:45.105422 10200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:45.119957 10205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:45.119999 10209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:45.120035 10207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:45.121354 10200 server_base.cc:1034] running on GCE node
I20260807 22:48:45.122148 10200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:45.123401 10200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:45.124703 10200 hybrid_clock.cc:648] HybridClock initialized: now 1786142925124646 us; error 45 us; skew 500 ppm
I20260807 22:48:45.125159 10200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 22:48:45.134327 10200 webserver.cc:460] Webserver started at http://127.20.10.131:33433/ using document root <none> and password file <none>
I20260807 22:48:45.135675 10200 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:45.135869 10200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:45.136391 10200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:45.151754 10200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data/instance:
uuid: "08f7b23411a742578a5e3b8102b39dee"
format_stamp: "Formatted at 2026-08-07 22:48:45 on dist-test-slave-csg5"
server_key: "e9c6e98d621d07abb5b5d0bbcca59b4d"
server_key_iv: "fef8372b33bc28dc1246852c70f483b6"
server_key_version: "encryptionkey@0"
I20260807 22:48:45.153081 10200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance:
uuid: "08f7b23411a742578a5e3b8102b39dee"
format_stamp: "Formatted at 2026-08-07 22:48:45 on dist-test-slave-csg5"
server_key: "e9c6e98d621d07abb5b5d0bbcca59b4d"
server_key_iv: "fef8372b33bc28dc1246852c70f483b6"
server_key_version: "encryptionkey@0"
I20260807 22:48:45.161818 10200 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 22:48:45.168032 10215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:45.170249 10200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 22:48:45.170491 10200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal
uuid: "08f7b23411a742578a5e3b8102b39dee"
format_stamp: "Formatted at 2026-08-07 22:48:45 on dist-test-slave-csg5"
server_key: "e9c6e98d621d07abb5b5d0bbcca59b4d"
server_key_iv: "fef8372b33bc28dc1246852c70f483b6"
server_key_version: "encryptionkey@0"
I20260807 22:48:45.170849 10200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:45.171766 10053 heartbeater.cc:503] Master 127.20.10.190:33641 was elected leader, sending a full tablet report...
I20260807 22:48:45.198532 10200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:45.199812 10200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:45.200299 10200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:45.201658 10200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:48:45.203801 10200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:48:45.203959 10200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:45.204124 10200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:48:45.204325 10200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:45.241921 10200 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.131:41489
I20260807 22:48:45.242017 10335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.131:41489 every 8 connection(s)
I20260807 22:48:45.243579 10200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/data/info.pb
I20260807 22:48:45.243928 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10200
I20260807 22:48:45.244115 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal/instance
I20260807 22:48:45.244611 20522 external_mini_cluster.cc:1428] Setting key c3ecc3a748372d819f9ffa91e68fb167
I20260807 22:48:45.259647 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.247589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.131:37351 (local address 127.20.10.190:33641)
0807 22:48:45.248286 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.248291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.249523 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 22:48:45.251534 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.251540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.251673 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.251782 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:45.253253 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.254127 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.256052 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.256634 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.256757 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.258425 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:45.258430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:45.258433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:45.258454 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:45.258497 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:45.258510 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:45.258621 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.258987 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.259104 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":478,"threads_started":1}
I20260807 22:48:45.260725 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.248408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:33641 (local address 127.20.10.131:37351)
0807 22:48:45.249360 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.249407 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.251176 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.252071 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.252082 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.252192 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:48:45.252973 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.252995 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.254279 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.254283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.255895 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.255909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.256938 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.256946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.257165 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.257966 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:48:45.257969 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:48:45.258015 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:48:45.258129 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:48:45.258621 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:48:45.258636 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:48:45.258646 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.258957 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.259420 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":298,"threads_started":1}
I20260807 22:48:45.262251 10336 heartbeater.cc:347] Connected to a master server at 127.20.10.190:33641
I20260807 22:48:45.262596 10336 heartbeater.cc:464] Registering TS with master...
I20260807 22:48:45.263461 10336 heartbeater.cc:511] Master 127.20.10.190:33641 requested a full tablet report, sending...
I20260807 22:48:45.265162 9850 ts_manager.cc:194] Registered new tserver with Master: 08f7b23411a742578a5e3b8102b39dee (127.20.10.131:41489)
I20260807 22:48:45.266070 9850 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.10.131:37351
I20260807 22:48:45.274410 20522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:48:45.286587 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.279422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.20.10.190:33641)
0807 22:48:45.279745 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.279755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.280151 (+ 396us) server_negotiation.cc:407] Connection header received
0807 22:48:45.280403 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.280407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.280483 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.280603 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:45.281183 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.282012 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.283732 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.284225 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.284382 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.285743 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:45.285748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:45.285752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:45.285776 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:45.285821 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:45.285836 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:45.285949 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.286127 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.286246 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 22:48:45.291550 9850 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:48:45.299019 9850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:48:45.334730 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.313403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:41489 (local address 127.0.0.1:46598)
0807 22:48:45.314970 (+ 1567us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.315030 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.315264 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.319115 (+ 3851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.319127 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.319175 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:45.319755 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.319774 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.327092 (+ 7318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.327101 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.331289 (+ 4188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.331307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.333407 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.333414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.333697 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.333813 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.334010 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.334273 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1147,"mutex_wait_us":1,"thread_start_us":1016,"threads_started":1}
I20260807 22:48:45.334730 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.314121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.20.10.131:41489)
0807 22:48:45.317990 (+ 3869us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.318003 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.318077 (+ 74us) server_negotiation.cc:407] Connection header received
0807 22:48:45.318241 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.318255 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.318755 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.320037 (+ 1282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:45.322040 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.324365 (+ 2325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.331612 (+ 7247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.333188 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.333376 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.333716 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.334071 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.334272 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1857,"thread_start_us":1622,"threads_started":1}
I20260807 22:48:45.339236 10268 tablet_service.cc:1467] Processing CreateTablet for tablet 7e204ddccb054dd2b8c004681458288a (DEFAULT_TABLE table=test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:45.340363 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.315949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.129:36455 (local address 127.0.0.1:51064)
0807 22:48:45.329924 (+ 13975us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.329959 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.330140 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.332229 (+ 2089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.332235 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.332267 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:45.332604 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.332617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.334779 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.334784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.337343 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.337363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.339446 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.339452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.339684 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.339731 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.339871 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.340025 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13778,"thread_start_us":14323,"threads_started":1}
I20260807 22:48:45.340924 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.316429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.20.10.129:36455)
0807 22:48:45.331053 (+ 14624us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.331062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.331100 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:48:45.331252 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.331262 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.331629 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.331893 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:45.332832 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.334546 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.337632 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.339223 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.339376 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.339682 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.339934 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.340235 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14435,"thread_start_us":14285,"threads_started":1}
I20260807 22:48:45.341027 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e204ddccb054dd2b8c004681458288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:45.342160 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.313692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:37449 (local address 127.0.0.1:47784)
0807 22:48:45.326681 (+ 12989us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.326709 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.326886 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.333503 (+ 6617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.333509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.333539 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:45.334094 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.334109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.336211 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.336217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.339137 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.339158 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.341324 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.341332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.341501 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.341533 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.341640 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.341786 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12765,"mutex_wait_us":74,"thread_start_us":11007,"threads_started":1}
I20260807 22:48:45.342227 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.314137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.20.10.130:37449)
0807 22:48:45.332534 (+ 18397us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.332542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.332579 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:48:45.332664 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.332670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.332990 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.333228 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:45.334302 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.336027 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.339383 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.340992 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.341153 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.341389 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.341708 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.341881 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18218,"thread_start_us":18094,"threads_started":1}
I20260807 22:48:45.345029 9983 tablet_service.cc:1467] Processing CreateTablet for tablet 7e204ddccb054dd2b8c004681458288a (DEFAULT_TABLE table=test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:45.346720 9983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e204ddccb054dd2b8c004681458288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:45.346390 10124 tablet_service.cc:1467] Processing CreateTablet for tablet 7e204ddccb054dd2b8c004681458288a (DEFAULT_TABLE table=test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:48:45.347903 10124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e204ddccb054dd2b8c004681458288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:45.356256 10356 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Bootstrap starting.
I20260807 22:48:45.360160 10356 tablet_bootstrap.cc:654] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:45.361286 10356 log.cc:824] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:45.363276 10358 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Bootstrap starting.
I20260807 22:48:45.364760 10359 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Bootstrap starting.
I20260807 22:48:45.364916 10356 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: No bootstrap required, opened a new log
I20260807 22:48:45.365242 10356 ts_tablet_manager.cc:1397] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 22:48:45.369048 10358 tablet_bootstrap.cc:654] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:45.369048 10359 tablet_bootstrap.cc:654] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:45.370565 10359 log.cc:824] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:45.370565 10358 log.cc:824] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:45.374425 10358 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: No bootstrap required, opened a new log
I20260807 22:48:45.374527 10359 tablet_bootstrap.cc:492] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: No bootstrap required, opened a new log
I20260807 22:48:45.374835 10358 ts_tablet_manager.cc:1397] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260807 22:48:45.374899 10359 ts_tablet_manager.cc:1397] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 22:48:45.374590 10356 raft_consensus.cc:348] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.375181 10356 raft_consensus.cc:374] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:45.375314 10356 raft_consensus.cc:729] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f7b23411a742578a5e3b8102b39dee, State: Initialized, Role: FOLLOWER
I20260807 22:48:45.376111 10356 consensus_queue.cc:260] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.377630 10336 heartbeater.cc:503] Master 127.20.10.190:33641 was elected leader, sending a full tablet report...
I20260807 22:48:45.378131 10356 ts_tablet_manager.cc:1428] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Time spent starting tablet: real 0.013s user 0.003s sys 0.010s
I20260807 22:48:45.385792 10359 raft_consensus.cc:348] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.385963 10358 raft_consensus.cc:348] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.386291 10359 raft_consensus.cc:374] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:45.386391 10358 raft_consensus.cc:374] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:45.386436 10359 raft_consensus.cc:729] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f33391fc79147caa42a2697197ba668, State: Initialized, Role: FOLLOWER
I20260807 22:48:45.386499 10358 raft_consensus.cc:729] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13069874cf014e5f9c8450f1f5aed42c, State: Initialized, Role: FOLLOWER
I20260807 22:48:45.387205 10358 consensus_queue.cc:260] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.387209 10359 consensus_queue.cc:260] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.389113 10195 heartbeater.cc:503] Master 127.20.10.190:33641 was elected leader, sending a full tablet report...
I20260807 22:48:45.389286 10359 ts_tablet_manager.cc:1428] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 22:48:45.389837 10358 ts_tablet_manager.cc:1428] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260807 22:48:45.395890 10054 tablet.cc:2369] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:45.437849 10365 raft_consensus.cc:482] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:48:45.438267 10365 raft_consensus.cc:504] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.440342 10365 leader_election.cc:288] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08f7b23411a742578a5e3b8102b39dee (127.20.10.131:41489), 13069874cf014e5f9c8450f1f5aed42c (127.20.10.130:37449)
W20260807 22:48:45.448156 10196 tablet.cc:2369] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:45.450176 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.440927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:46447 (local address 127.20.10.130:37449)
0807 22:48:45.441153 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.441161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.442043 (+ 882us) server_negotiation.cc:407] Connection header received
0807 22:48:45.442376 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.442382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.442474 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.442566 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:45.443471 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.444599 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.448191 (+ 3592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.449107 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.449302 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.449383 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.449758 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.449910 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:48:45.450320 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.441401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.10.129:34261 (local address 127.20.10.131:41489)
0807 22:48:45.441619 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.441625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.442148 (+ 523us) server_negotiation.cc:407] Connection header received
0807 22:48:45.442332 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.442338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.442423 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.442537 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:48:45.443471 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.444599 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.448144 (+ 3545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.449401 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.449539 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.449588 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.449931 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.450071 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:48:45.450330 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.440765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.131:41489 (local address 127.20.10.129:34261)
0807 22:48:45.442045 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.442062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.442189 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.442591 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.442596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.442746 (+ 150us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:45.443289 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.443302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.444775 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.444781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.447853 (+ 3072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.447879 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.449596 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.449602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.449760 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.449789 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.449878 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.449995 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"mutex_wait_us":172,"spinlock_wait_cycles":27392,"thread_start_us":851,"threads_started":1}
I20260807 22:48:45.450749 10367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.440611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.130:37449 (local address 127.20.10.129:46447)
0807 22:48:45.441895 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0807 22:48:45.441920 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:48:45.442099 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:48:45.442591 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:48:45.442596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:48:45.442624 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:48:45.443189 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.443209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.444775 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.444781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.447877 (+ 3096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:48:45.447886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.449327 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.449334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:48:45.449533 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.449583 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 22:48:45.449685 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:48:45.449838 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":308,"spinlock_wait_cycles":24832,"thread_start_us":835,"threads_started":1}
I20260807 22:48:45.451936 10289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e204ddccb054dd2b8c004681458288a" candidate_uuid: "1f33391fc79147caa42a2697197ba668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f7b23411a742578a5e3b8102b39dee" is_pre_election: true
I20260807 22:48:45.451977 10146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e204ddccb054dd2b8c004681458288a" candidate_uuid: "1f33391fc79147caa42a2697197ba668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13069874cf014e5f9c8450f1f5aed42c" is_pre_election: true
I20260807 22:48:45.452670 10146 raft_consensus.cc:2454] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f33391fc79147caa42a2697197ba668 in term 0.
I20260807 22:48:45.452670 10289 raft_consensus.cc:2454] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f33391fc79147caa42a2697197ba668 in term 0.
I20260807 22:48:45.453514 9933 leader_election.cc:302] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08f7b23411a742578a5e3b8102b39dee, 13069874cf014e5f9c8450f1f5aed42c, 1f33391fc79147caa42a2697197ba668; no voters:
I20260807 22:48:45.454007 10365 raft_consensus.cc:2781] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:48:45.454172 10365 raft_consensus.cc:482] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:48:45.454298 10365 raft_consensus.cc:3037] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:45.456851 10365 raft_consensus.cc:504] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.457541 10365 leader_election.cc:288] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [CANDIDATE]: Term 1 election: Requested vote from peers 08f7b23411a742578a5e3b8102b39dee (127.20.10.131:41489), 13069874cf014e5f9c8450f1f5aed42c (127.20.10.130:37449)
I20260807 22:48:45.458266 10289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e204ddccb054dd2b8c004681458288a" candidate_uuid: "1f33391fc79147caa42a2697197ba668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f7b23411a742578a5e3b8102b39dee"
I20260807 22:48:45.458341 10146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e204ddccb054dd2b8c004681458288a" candidate_uuid: "1f33391fc79147caa42a2697197ba668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13069874cf014e5f9c8450f1f5aed42c"
I20260807 22:48:45.458559 10289 raft_consensus.cc:3037] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:45.458621 10146 raft_consensus.cc:3037] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:45.461153 10289 raft_consensus.cc:2454] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f33391fc79147caa42a2697197ba668 in term 1.
I20260807 22:48:45.461653 9933 leader_election.cc:302] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08f7b23411a742578a5e3b8102b39dee, 1f33391fc79147caa42a2697197ba668; no voters:
I20260807 22:48:45.461668 10146 raft_consensus.cc:2454] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f33391fc79147caa42a2697197ba668 in term 1.
I20260807 22:48:45.462065 10365 raft_consensus.cc:2781] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:45.462730 10365 raft_consensus.cc:686] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 1 LEADER]: Becoming Leader. State: Replica: 1f33391fc79147caa42a2697197ba668, State: Running, Role: LEADER
I20260807 22:48:45.463300 10365 consensus_queue.cc:237] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } }
I20260807 22:48:45.469056 9848 catalog_manager.cc:5495] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f33391fc79147caa42a2697197ba668 (127.20.10.129). New cstate: current_term: 1 leader_uuid: "1f33391fc79147caa42a2697197ba668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f33391fc79147caa42a2697197ba668" member_type: VOTER last_known_addr { host: "127.20.10.129" port: 36455 } health_report { overall_health: HEALTHY } } }
W20260807 22:48:45.496837 10337 tablet.cc:2369] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:48:45.523406 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.517369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.20.10.129:36455)
0807 22:48:45.517651 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.517659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.517892 (+ 233us) server_negotiation.cc:407] Connection header received
0807 22:48:45.518157 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.518161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.518269 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.518437 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:48:45.518917 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.519662 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.521035 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.521489 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.521644 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.522179 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:48:45.522819 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:48:45.522987 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.523046 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.523135 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 22:48:45.532991 10289 raft_consensus.cc:1261] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 1 FOLLOWER]: Refusing update from remote peer 1f33391fc79147caa42a2697197ba668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:45.532991 10146 raft_consensus.cc:1261] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 1 FOLLOWER]: Refusing update from remote peer 1f33391fc79147caa42a2697197ba668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:48:45.534212 10365 consensus_queue.cc:1035] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13069874cf014e5f9c8450f1f5aed42c" member_type: VOTER last_known_addr { host: "127.20.10.130" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:45.534776 10369 consensus_queue.cc:1035] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08f7b23411a742578a5e3b8102b39dee" member_type: VOTER last_known_addr { host: "127.20.10.131" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:48:45.565685 9848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:45.566084 9848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:48:45.570557 9848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83935a64cc874b5291eccfffc3b18424: Sending DeleteTablet for 3 replicas of tablet 7e204ddccb054dd2b8c004681458288a
I20260807 22:48:45.572167 10124 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e204ddccb054dd2b8c004681458288a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:45 UTC) from {username='slave'} at 127.0.0.1:47784
I20260807 22:48:45.572167 10268 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e204ddccb054dd2b8c004681458288a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:45 UTC) from {username='slave'} at 127.0.0.1:46598
I20260807 22:48:45.572407 9983 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e204ddccb054dd2b8c004681458288a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:48:45 UTC) from {username='slave'} at 127.0.0.1:51064
I20260807 22:48:45.577250 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9915
I20260807 22:48:45.585616 10386 tablet_replica.cc:330] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: stopping tablet replica
I20260807 22:48:45.587265 10386 raft_consensus.cc:2229] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:48:45.589179 10386 raft_consensus.cc:2258] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:45.590469 10385 tablet_replica.cc:330] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: stopping tablet replica
I20260807 22:48:45.591130 10385 raft_consensus.cc:2229] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:45.591791 10385 raft_consensus.cc:2258] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:45.592736 10384 tablet_replica.cc:330] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: stopping tablet replica
I20260807 22:48:45.593212 10386 ts_tablet_manager.cc:1905] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:45.593259 10384 raft_consensus.cc:2229] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:48:45.593832 10384 raft_consensus.cc:2258] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:48:45.596437 10384 ts_tablet_manager.cc:1905] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:45.598985 10385 ts_tablet_manager.cc:1905] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:48:45.599952 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.580045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.20.10.129:36455)
0807 22:48:45.580345 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.580355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.589599 (+ 9244us) server_negotiation.cc:407] Connection header received
0807 22:48:45.589895 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.589903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.590046 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.590164 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:45.590950 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.592138 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.594956 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.595870 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.596022 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.597813 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:45.597826 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:45.597848 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:45.598839 (+ 991us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:45.598908 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:45.598936 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:45.599061 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.599372 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.599514 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 22:48:45.601509 10047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:45.607311 10385 ts_tablet_manager.cc:1918] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:45.607523 10385 log.cc:1196] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-2/wal/wals/7e204ddccb054dd2b8c004681458288a
I20260807 22:48:45.608143 10385 ts_tablet_manager.cc:1939] T 7e204ddccb054dd2b8c004681458288a P 08f7b23411a742578a5e3b8102b39dee: Deleting consensus metadata
I20260807 22:48:45.610268 10384 ts_tablet_manager.cc:1918] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:45.610488 10384 log.cc:1196] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-1/wal/wals/7e204ddccb054dd2b8c004681458288a
I20260807 22:48:45.610561 9832 catalog_manager.cc:4841] TS 08f7b23411a742578a5e3b8102b39dee (127.20.10.131:41489): tablet 7e204ddccb054dd2b8c004681458288a (table test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]) successfully deleted
I20260807 22:48:45.611122 10384 ts_tablet_manager.cc:1939] T 7e204ddccb054dd2b8c004681458288a P 13069874cf014e5f9c8450f1f5aed42c: Deleting consensus metadata
I20260807 22:48:45.612958 9834 catalog_manager.cc:4841] TS 13069874cf014e5f9c8450f1f5aed42c (127.20.10.130:37449): tablet 7e204ddccb054dd2b8c004681458288a (table test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]) successfully deleted
I20260807 22:48:45.712411 10386 ts_tablet_manager.cc:1918] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:48:45.712719 10386 log.cc:1196] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Deleting WAL directory at /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142560216906-20522-0/minicluster-data/ts-0/wal/wals/7e204ddccb054dd2b8c004681458288a
I20260807 22:48:45.713347 10386 ts_tablet_manager.cc:1939] T 7e204ddccb054dd2b8c004681458288a P 1f33391fc79147caa42a2697197ba668: Deleting consensus metadata
I20260807 22:48:45.715600 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9915
I20260807 22:48:45.715569 9832 catalog_manager.cc:4841] TS 1f33391fc79147caa42a2697197ba668 (127.20.10.129:36455): tablet 7e204ddccb054dd2b8c004681458288a (table test-table [id=3ffa6047bee84aadaeaa6422c609e5e2]) successfully deleted
I20260807 22:48:45.743559 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10057
I20260807 22:48:45.752543 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.744488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.20.10.130:37449)
0807 22:48:45.744720 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.744726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.744748 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:48:45.744953 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.744957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.745067 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.745201 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:45.745643 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.746629 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.748278 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.748743 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.748894 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.750423 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:45.750434 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:45.750451 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:45.751286 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:45.751342 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:45.751365 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:45.751596 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.751868 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.751994 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:48:45.753451 10190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:45.852789 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10057
I20260807 22:48:45.878196 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10200
I20260807 22:48:45.886251 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:45.879052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.20.10.131:41489)
0807 22:48:45.879261 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:45.879266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:45.879343 (+ 77us) server_negotiation.cc:407] Connection header received
0807 22:48:45.879571 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:45.879576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:45.879685 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:45.879777 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:45.880172 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.880967 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.882507 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:45.883127 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:45.883337 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:45.884503 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:45.884513 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:45.884529 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:45.885237 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:45.885299 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:45.885323 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:45.885432 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:45.885604 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 22:48:45.885722 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 22:48:45.887202 10330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:45.983254 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10200
I20260807 22:48:46.005609 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 9815
I20260807 22:48:46.007107 9882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:46.123596 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 9815
2026-08-07T22:48:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3221 ms)
[----------] 6 tests from EncryptionPolicyTest (25484 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:46 dist-test-slave-csg5 krb5kdc[10396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:46 dist-test-slave-csg5 krb5kdc[10396](info): set up 2 sockets
Aug 07 22:48:46 dist-test-slave-csg5 krb5kdc[10396](info): commencing operation
krb5kdc: starting...
W20260807 22:48:47.397871 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.003s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:47 dist-test-slave-csg5 krb5kdc[10396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:47.791115 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35393
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:48.165378 10409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:48.165956 10409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:48.166157 10409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:48.173995 10409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:48.174139 10409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:48.174190 10409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:48.174300 10409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:48.174376 10409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:48.186268 10409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35393
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:48.187954 10409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:48.189491 10409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:48.204486 10416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:48.204427 10415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:48.204557 10420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:48.209122 10409 server_base.cc:1034] running on GCE node
I20260807 22:48:48.210316 10409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:48.212481 10409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:48.213815 10409 hybrid_clock.cc:648] HybridClock initialized: now 1786142928213762 us; error 57 us; skew 500 ppm
I20260807 22:48:48.214316 10409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:48 dist-test-slave-csg5 krb5kdc[10396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142928, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:48.224478 10409 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:48.227222 10409 webserver.cc:460] Webserver started at http://127.0.0.1:33447/ using document root <none> and password file <none>
I20260807 22:48:48.228284 10409 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:48.228478 10409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:48.228852 10409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:48.232532 10409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "27895ad05e674cb99c54851c609119b9"
format_stamp: "Formatted at 2026-08-07 22:48:48 on dist-test-slave-csg5"
server_key: "43d4fa8bcb8a8ec999e22dee43010079"
server_key_iv: "952a3a205854ff34fefd045b2215d759"
server_key_version: "encryptionkey@0"
I20260807 22:48:48.233469 10409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "27895ad05e674cb99c54851c609119b9"
format_stamp: "Formatted at 2026-08-07 22:48:48 on dist-test-slave-csg5"
server_key: "43d4fa8bcb8a8ec999e22dee43010079"
server_key_iv: "952a3a205854ff34fefd045b2215d759"
server_key_version: "encryptionkey@0"
I20260807 22:48:48.240201 10409 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:48:48.244503 10427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:48.246138 10409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:48:48.246388 10409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "27895ad05e674cb99c54851c609119b9"
format_stamp: "Formatted at 2026-08-07 22:48:48 on dist-test-slave-csg5"
server_key: "43d4fa8bcb8a8ec999e22dee43010079"
server_key_iv: "952a3a205854ff34fefd045b2215d759"
server_key_version: "encryptionkey@0"
I20260807 22:48:48.246639 10409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:48.263670 10409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:48.268625 10409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:48.268971 10409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:48.291661 10409 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35393
I20260807 22:48:48.291707 10484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35393 every 8 connection(s)
I20260807 22:48:48.293385 10409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:48.297688 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:48.300000 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10409
I20260807 22:48:48.300278 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:48.301123 20522 external_mini_cluster.cc:1428] Setting key 69fed0a1e1a0a4e3b3c807c4692b2a53
I20260807 22:48:48.310881 10485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9: Bootstrap starting.
I20260807 22:48:48.314604 10485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:48.315837 10485 log.cc:824] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:48:48 dist-test-slave-csg5 krb5kdc[10396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:48:48.319466 10485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9: No bootstrap required, opened a new log
I20260807 22:48:48.329342 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:48.303493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52053 (local address 127.0.0.1:35393)
0807 22:48:48.304534 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:48.304551 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:48.304596 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:48:48.306444 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:48.306470 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:48.306987 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:48.307389 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:48.309027 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:48.310548 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:48.312470 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:48.313255 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:48.313494 (+ 239us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:48.318637 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:48.318650 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:48.318691 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:48.318741 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:48.323133 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:48.324058 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:48.324067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:48.324076 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:48.324330 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:48.324817 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:48.324822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:48.324826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:48.325918 (+ 1092us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:48.326286 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:48.327272 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 22:48:48.327713 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":435,"threads_started":1}
I20260807 22:48:48.331189 10485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } }
I20260807 22:48:48.331749 10485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:48.331873 10485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27895ad05e674cb99c54851c609119b9, State: Initialized, Role: FOLLOWER
I20260807 22:48:48.332516 10485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } }
I20260807 22:48:48.332824 10485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:48.332985 10485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:48.333161 10485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:48.334620 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:48:48.335991 10485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } }
I20260807 22:48:48.336525 10485 leader_election.cc:302] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27895ad05e674cb99c54851c609119b9; no voters:
I20260807 22:48:48.337177 10485 leader_election.cc:288] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:48.337396 10490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:48.344194 10490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [term 1 LEADER]: Becoming Leader. State: Replica: 27895ad05e674cb99c54851c609119b9, State: Running, Role: LEADER
I20260807 22:48:48.345053 10490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } }
I20260807 22:48:48.345642 10485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:48.349171 10495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27895ad05e674cb99c54851c609119b9. Latest consensus state: current_term: 1 leader_uuid: "27895ad05e674cb99c54851c609119b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } } }
I20260807 22:48:48.349570 10495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:48.350186 10493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27895ad05e674cb99c54851c609119b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27895ad05e674cb99c54851c609119b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35393 } } }
I20260807 22:48:48.350548 10493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:48.361944 10505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:48.369005 10505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:48.376329 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:48.347830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43725 (local address 127.0.0.1:35393)
0807 22:48:48.348016 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:48.348027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:48.352120 (+ 4093us) server_negotiation.cc:407] Connection header received
0807 22:48:48.352688 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:48.352698 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:48.352801 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:48.352896 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:48.355971 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:48.357473 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:48.360047 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:48.360898 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:48.361048 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:48.365250 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:48.365260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:48.365268 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:48.365328 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:48.370507 (+ 5179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:48.372167 (+ 1660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:48.372176 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:48.372183 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:48.372406 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:48.373750 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:48.373755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:48.373758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:48.374269 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:48.374566 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:48.375705 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0807 22:48:48.375931 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:48:48.377204 10517 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:48:48.380000 10505 catalog_manager.cc:1347] Generated new cluster ID: 60bb230e87664d288eb7aa01d57a7c67
I20260807 22:48:48.380122 10505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:48.405805 10505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:48.407513 10505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:48.424220 10505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27895ad05e674cb99c54851c609119b9: Generated new TSK 0
I20260807 22:48:48.425464 10505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:48.438150 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10409
I20260807 22:48:48.440858 10479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:48.542495 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10409
W20260807 22:48:48.565053 10392 connection.cc:535] client connection to 127.0.0.1:35393 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35393): Connection reset by peer (error 104)
2026-08-07T22:48:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:48 dist-test-slave-csg5 krb5kdc[10522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:48 dist-test-slave-csg5 krb5kdc[10522](info): set up 2 sockets
Aug 07 22:48:48 dist-test-slave-csg5 krb5kdc[10522](info): commencing operation
krb5kdc: starting...
W20260807 22:48:49.787000 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:50 dist-test-slave-csg5 krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:50.176548 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:50.582942 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:50.583287 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:50.583388 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:50.591990 10538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:50.592141 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:50.592217 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:50.592281 10538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:50.592330 10538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:50.603161 10538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:50.604583 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:50.606057 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:50.620965 10543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:50.620936 10544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:50.620916 10546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:50.622161 10538 server_base.cc:1034] running on GCE node
I20260807 22:48:50.623456 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:50.625229 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:50.626506 10538 hybrid_clock.cc:648] HybridClock initialized: now 1786142930626456 us; error 49 us; skew 500 ppm
I20260807 22:48:50.626971 10538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:50 dist-test-slave-csg5 krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142930, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:50.630985 10538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:50.632817 10538 webserver.cc:460] Webserver started at http://127.0.0.1:39743/ using document root <none> and password file <none>
I20260807 22:48:50.633658 10538 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:50.633837 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:50.634223 10538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:50.637774 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "90ea662b87274f91b28a2bf49365420f"
format_stamp: "Formatted at 2026-08-07 22:48:50 on dist-test-slave-csg5"
server_key: "ce5eed777e7cf21d8bd1c33a4c728e77"
server_key_iv: "e9b47ae23371a379713ded82dcf0fb8d"
server_key_version: "encryptionkey@0"
I20260807 22:48:50.638783 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "90ea662b87274f91b28a2bf49365420f"
format_stamp: "Formatted at 2026-08-07 22:48:50 on dist-test-slave-csg5"
server_key: "ce5eed777e7cf21d8bd1c33a4c728e77"
server_key_iv: "e9b47ae23371a379713ded82dcf0fb8d"
server_key_version: "encryptionkey@0"
I20260807 22:48:50.644659 10538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 22:48:50.648696 10554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:50.650336 10538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:48:50.650558 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "90ea662b87274f91b28a2bf49365420f"
format_stamp: "Formatted at 2026-08-07 22:48:50 on dist-test-slave-csg5"
server_key: "ce5eed777e7cf21d8bd1c33a4c728e77"
server_key_iv: "e9b47ae23371a379713ded82dcf0fb8d"
server_key_version: "encryptionkey@0"
I20260807 22:48:50.650861 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:50.676575 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:50.682916 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:50.683324 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:50.707966 10538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42617
I20260807 22:48:50.707993 10615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42617 every 8 connection(s)
I20260807 22:48:50.709704 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:50.714548 10616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:50.718329 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10538
I20260807 22:48:50.718578 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:50.719158 20522 external_mini_cluster.cc:1428] Setting key e474c75d5456d837a1fbe9106658a45d
I20260807 22:48:50.729892 10616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f: Bootstrap starting.
I20260807 22:48:50.734287 10616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:50.735615 10616 log.cc:824] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:50.736449 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:50.721793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43747 (local address 127.0.0.1:42617)
0807 22:48:50.722769 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:50.722781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:50.722827 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:48:50.724613 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:50.724640 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:50.725138 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:50.725516 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:50.726610 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:50.728325 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:50.730788 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:50.731577 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:50.731747 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:50.733337 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:50.733347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:50.733363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:50.733992 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:50.734081 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:50.734111 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:50.734245 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:50.734860 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 22:48:50.735077 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":458,"threads_started":1}
I20260807 22:48:50.738963 10616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f: No bootstrap required, opened a new log
I20260807 22:48:50.741392 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:48:50.747931 10616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } }
I20260807 22:48:50.748521 10616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:50.748656 10616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ea662b87274f91b28a2bf49365420f, State: Initialized, Role: FOLLOWER
I20260807 22:48:50.749346 10616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } }
I20260807 22:48:50.749640 10616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:50.749900 10616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:50.750111 10616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:50.753504 10616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } }
I20260807 22:48:50.754211 10616 leader_election.cc:302] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90ea662b87274f91b28a2bf49365420f; no voters:
I20260807 22:48:50.754954 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:50.746476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42731 (local address 127.0.0.1:42617)
0807 22:48:50.746762 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:50.746770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:50.747133 (+ 363us) server_negotiation.cc:407] Connection header received
0807 22:48:50.747363 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:50.747367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:50.747440 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:50.747533 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:50.748260 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:50.749567 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:50.751660 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:50.752288 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:50.752412 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:50.753837 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:50.753842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:50.753846 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:50.753868 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:50.753910 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:50.753927 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:50.754031 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:50.754407 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 22:48:50.754473 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:48:50.755172 10616 leader_election.cc:288] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:50.755389 10627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:50.757227 10627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [term 1 LEADER]: Becoming Leader. State: Replica: 90ea662b87274f91b28a2bf49365420f, State: Running, Role: LEADER
I20260807 22:48:50.758016 10616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:50.757880 10627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } }
I20260807 22:48:50.761467 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90ea662b87274f91b28a2bf49365420f. Latest consensus state: current_term: 1 leader_uuid: "90ea662b87274f91b28a2bf49365420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } } }
I20260807 22:48:50.761564 10628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90ea662b87274f91b28a2bf49365420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ea662b87274f91b28a2bf49365420f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42617 } } }
I20260807 22:48:50.762001 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:50.762003 10628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:50.764814 10633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:50.771412 10633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:50.781965 10633 catalog_manager.cc:1347] Generated new cluster ID: 6aa6d6b5a3314872a00263a3297729da
I20260807 22:48:50.782100 10633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:50.794862 10633 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:50.797159 10633 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:50.810170 10633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90ea662b87274f91b28a2bf49365420f: Generated new TSK 0
I20260807 22:48:50.811473 10633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:50.827622 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10538
W20260807 22:48:50.829062 10611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43747
W20260807 22:48:50.830207 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10538: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:48:50.830488 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10538
W20260807 22:48:50.854444 10520 connection.cc:535] client connection to 127.0.0.1:42617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42617): Connection reset by peer (error 104)
2026-08-07T22:48:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:50 dist-test-slave-csg5 krb5kdc[10650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:50 dist-test-slave-csg5 krb5kdc[10650](info): set up 2 sockets
Aug 07 22:48:50 dist-test-slave-csg5 krb5kdc[10650](info): commencing operation
krb5kdc: starting...
W20260807 22:48:52.118119 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.003s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:52 dist-test-slave-csg5 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:52.531996 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:52.911326 10665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:52.911666 10665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:52.911787 10665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:52.919587 10665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:52.919746 10665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:52.919796 10665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:52.919852 10665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:52.919898 10665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:52.930588 10665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:52.932060 10665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:52.933478 10665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:52.947026 10674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:52.947096 10671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:52.947297 10672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:52.947988 10665 server_base.cc:1034] running on GCE node
I20260807 22:48:52.948966 10665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:52.950819 10665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:52.952128 10665 hybrid_clock.cc:648] HybridClock initialized: now 1786142932952081 us; error 57 us; skew 500 ppm
I20260807 22:48:52.952595 10665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:52 dist-test-slave-csg5 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142932, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:52.956473 10665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:52.958220 10665 webserver.cc:460] Webserver started at http://127.0.0.1:43607/ using document root <none> and password file <none>
I20260807 22:48:52.959065 10665 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:52.959241 10665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:52.959592 10665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:52.963054 10665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "bee28aec16cc4bffaee7cb8b94ec49b0"
format_stamp: "Formatted at 2026-08-07 22:48:52 on dist-test-slave-csg5"
server_key: "5a39a97446f43aca60f4e6166d53ffb4"
server_key_iv: "904d228f8cedff047d568c67914f1156"
server_key_version: "encryptionkey@0"
I20260807 22:48:52.963956 10665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "bee28aec16cc4bffaee7cb8b94ec49b0"
format_stamp: "Formatted at 2026-08-07 22:48:52 on dist-test-slave-csg5"
server_key: "5a39a97446f43aca60f4e6166d53ffb4"
server_key_iv: "904d228f8cedff047d568c67914f1156"
server_key_version: "encryptionkey@0"
I20260807 22:48:52.970292 10665 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 22:48:52.974877 10682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:52.976724 10665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:48:52.976971 10665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "bee28aec16cc4bffaee7cb8b94ec49b0"
format_stamp: "Formatted at 2026-08-07 22:48:52 on dist-test-slave-csg5"
server_key: "5a39a97446f43aca60f4e6166d53ffb4"
server_key_iv: "904d228f8cedff047d568c67914f1156"
server_key_version: "encryptionkey@0"
I20260807 22:48:52.977240 10665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:52.983959 10665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:52.989019 10665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:52.989382 10665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:53.010270 10665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44009
I20260807 22:48:53.010289 10738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44009 every 8 connection(s)
I20260807 22:48:53.011885 10665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:53.016075 10739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:53.020855 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10665
I20260807 22:48:53.021101 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:53.021688 20522 external_mini_cluster.cc:1428] Setting key 7013835e6cde10e04adecc3c4779d59e
I20260807 22:48:53.028913 10739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0: Bootstrap starting.
I20260807 22:48:53.031548 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:53.023857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36209 (local address 127.0.0.1:44009)
0807 22:48:53.024812 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:53.024825 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:53.024869 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:48:53.027580 (+ 2711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:53.027611 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:53.028047 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:53.028525 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:53.029730 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:53.029736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:53.029751 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:53.029766 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:53.029834 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:53.029854 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:53.030106 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:53.030248 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:48:53.030468 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":470,"threads_started":1}
I20260807 22:48:53.032647 10739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:53.033859 10739 log.cc:824] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:53.035840 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:48:53.037931 10739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0: No bootstrap required, opened a new log
I20260807 22:48:53.044497 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:53.041435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33735 (local address 127.0.0.1:44009)
0807 22:48:53.041701 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:53.041711 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:53.042127 (+ 416us) server_negotiation.cc:407] Connection header received
0807 22:48:53.042313 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:53.042316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:53.042383 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:53.042480 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:53.043637 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:53.043642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:53.043646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:53.043666 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:53.043714 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:53.043736 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:53.043874 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:53.044031 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 22:48:53.044095 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:48:53.048550 10739 raft_consensus.cc:348] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } }
I20260807 22:48:53.048997 10739 raft_consensus.cc:374] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:53.049116 10739 raft_consensus.cc:729] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee28aec16cc4bffaee7cb8b94ec49b0, State: Initialized, Role: FOLLOWER
I20260807 22:48:53.049753 10739 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } }
I20260807 22:48:53.049996 10739 raft_consensus.cc:388] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:53.050194 10739 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:53.050367 10739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:53.053134 10739 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } }
I20260807 22:48:53.053710 10739 leader_election.cc:302] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bee28aec16cc4bffaee7cb8b94ec49b0; no voters:
I20260807 22:48:53.054476 10739 leader_election.cc:288] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:53.054646 10750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:53.056289 10750 raft_consensus.cc:686] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [term 1 LEADER]: Becoming Leader. State: Replica: bee28aec16cc4bffaee7cb8b94ec49b0, State: Running, Role: LEADER
I20260807 22:48:53.056895 10750 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } }
I20260807 22:48:53.057060 10739 sys_catalog.cc:564] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:53.060287 10751 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } } }
I20260807 22:48:53.060273 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee28aec16cc4bffaee7cb8b94ec49b0. Latest consensus state: current_term: 1 leader_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee28aec16cc4bffaee7cb8b94ec49b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44009 } } }
I20260807 22:48:53.060915 10751 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:53.060966 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:53.062719 10758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:53.069211 10758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:53.079231 10758 catalog_manager.cc:1347] Generated new cluster ID: 3df408cbb6b141428c2c44f4e8cb368b
I20260807 22:48:53.079398 10758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:53.097434 10758 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:53.097931 10758 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:53.104707 10758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bee28aec16cc4bffaee7cb8b94ec49b0: Generated new TSK 0
I20260807 22:48:53.105850 10758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:53.122694 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10665
W20260807 22:48:53.124418 10733 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36209
W20260807 22:48:53.125378 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:48:53.125584 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10665
2026-08-07T22:48:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:53 dist-test-slave-csg5 krb5kdc[10774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:53 dist-test-slave-csg5 krb5kdc[10774](info): set up 2 sockets
Aug 07 22:48:53 dist-test-slave-csg5 krb5kdc[10774](info): commencing operation
krb5kdc: starting...
W20260807 22:48:54.423467 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.001s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:54 dist-test-slave-csg5 krb5kdc[10774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:54.827112 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:55.198709 10789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:55.199249 10789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:55.199407 10789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:55.210860 10789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:55.211205 10789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:55.211356 10789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:55.211530 10789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:55.211615 10789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:55.228974 10789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:55.231328 10789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:55.232980 10789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:55.249158 10795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:55.249210 10798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:55.249187 10796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:55.253087 10789 server_base.cc:1034] running on GCE node
I20260807 22:48:55.254242 10789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:55.256043 10789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:55.257355 10789 hybrid_clock.cc:648] HybridClock initialized: now 1786142935257313 us; error 63 us; skew 500 ppm
I20260807 22:48:55.257797 10789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:55 dist-test-slave-csg5 krb5kdc[10774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142935, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:55.261722 10789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:55.263701 10789 webserver.cc:460] Webserver started at http://127.0.0.1:42609/ using document root <none> and password file <none>
I20260807 22:48:55.264506 10789 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:55.264667 10789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:55.265012 10789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:55.268386 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "1e253bb88d044b26a4bc1a68c3c8e908"
format_stamp: "Formatted at 2026-08-07 22:48:55 on dist-test-slave-csg5"
server_key: "07dc9208cde0f5c6783860ae663caf8d"
server_key_iv: "b102de3c996da0016eb05beb372e706b"
server_key_version: "encryptionkey@0"
I20260807 22:48:55.269269 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "1e253bb88d044b26a4bc1a68c3c8e908"
format_stamp: "Formatted at 2026-08-07 22:48:55 on dist-test-slave-csg5"
server_key: "07dc9208cde0f5c6783860ae663caf8d"
server_key_iv: "b102de3c996da0016eb05beb372e706b"
server_key_version: "encryptionkey@0"
I20260807 22:48:55.275388 10789 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 22:48:55.279663 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:55.281204 10789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:48:55.281448 10789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "1e253bb88d044b26a4bc1a68c3c8e908"
format_stamp: "Formatted at 2026-08-07 22:48:55 on dist-test-slave-csg5"
server_key: "07dc9208cde0f5c6783860ae663caf8d"
server_key_iv: "b102de3c996da0016eb05beb372e706b"
server_key_version: "encryptionkey@0"
I20260807 22:48:55.281701 10789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:55.304937 10789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:55.310736 10789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:55.311131 10789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:55.335162 10789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35931
I20260807 22:48:55.335202 10863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35931 every 8 connection(s)
I20260807 22:48:55.336838 10789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:55.341454 10864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:55.345942 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10789
I20260807 22:48:55.346166 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:55.346788 20522 external_mini_cluster.cc:1428] Setting key 2df6b822e7cadfec52124a844c1685a7
I20260807 22:48:55.355448 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908: Bootstrap starting.
I20260807 22:48:55.359671 10864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:55.361099 10864 log.cc:824] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:55.364898 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908: No bootstrap required, opened a new log
Aug 07 22:48:55 dist-test-slave-csg5 krb5kdc[10774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:48:55.376097 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:55.348941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37953 (local address 127.0.0.1:35931)
0807 22:48:55.349880 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:55.349899 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:55.349957 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:48:55.353382 (+ 3425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:55.353422 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:55.354064 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:55.354586 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:55.356329 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:55.358502 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:55.360144 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:55.361288 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:55.361480 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:55.366764 (+ 5284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:55.366774 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:55.366792 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:55.366857 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:55.370606 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:55.371847 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:55.371855 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:55.371869 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:55.372080 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:55.372617 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:55.372621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:55.372623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:55.373269 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:55.373558 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:55.374309 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 22:48:55.374908 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":425,"threads_started":1}
I20260807 22:48:55.376446 10864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260807 22:48:55.376950 10864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:55.377115 10864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e253bb88d044b26a4bc1a68c3c8e908, State: Initialized, Role: FOLLOWER
I20260807 22:48:55.377780 10864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260807 22:48:55.378095 10864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:55.378295 10864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:55.378521 10864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:55.380899 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:48:55.381846 10864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260807 22:48:55.382491 10864 leader_election.cc:302] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e253bb88d044b26a4bc1a68c3c8e908; no voters:
I20260807 22:48:55.383347 10864 leader_election.cc:288] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:55.383491 10869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:55.385233 10869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [term 1 LEADER]: Becoming Leader. State: Replica: 1e253bb88d044b26a4bc1a68c3c8e908, State: Running, Role: LEADER
I20260807 22:48:55.386068 10869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260807 22:48:55.386399 10864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:55.394337 10870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260807 22:48:55.395053 10870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:55.395668 10871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e253bb88d044b26a4bc1a68c3c8e908. Latest consensus state: current_term: 1 leader_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e253bb88d044b26a4bc1a68c3c8e908" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260807 22:48:55.397977 10871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:55.399454 10886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:55.407028 10886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:55.415668 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:55.389457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39125 (local address 127.0.0.1:35931)
0807 22:48:55.390999 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:55.391011 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:55.391744 (+ 733us) server_negotiation.cc:407] Connection header received
0807 22:48:55.391996 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:55.392005 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:55.392105 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:55.392293 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:55.394923 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:55.396815 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:55.398966 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:55.399989 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:55.400187 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:55.403583 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:55.403592 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:55.403601 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:48:55.403653 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:55.408828 (+ 5175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:55.410400 (+ 1572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:55.410415 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:55.410422 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:55.410709 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:48:55.411452 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:48:55.411465 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:48:55.411472 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:48:55.412181 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:55.412563 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:55.414559 (+ 1996us) server_negotiation.cc:299] Negotiation successful
0807 22:48:55.415008 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1339}
I20260807 22:48:55.425033 10886 catalog_manager.cc:1347] Generated new cluster ID: 005a04835cbf461fa82278a69fe8114d
I20260807 22:48:55.425309 10886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:55.450335 10886 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:55.452143 10886 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:55.458698 10886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e253bb88d044b26a4bc1a68c3c8e908: Generated new TSK 0
I20260807 22:48:55.459741 10886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:55.466395 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10789
I20260807 22:48:55.467758 10859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:48:55.569509 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10789
2026-08-07T22:48:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:55 dist-test-slave-csg5 krb5kdc[10899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:55 dist-test-slave-csg5 krb5kdc[10899](info): set up 2 sockets
Aug 07 22:48:55 dist-test-slave-csg5 krb5kdc[10899](info): commencing operation
krb5kdc: starting...
W20260807 22:48:56.841447 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:57 dist-test-slave-csg5 krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:57.235638 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:57.602725 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:57.603060 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:57.603232 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:57.610951 10916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:57.611083 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:57.611171 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:57.611310 10916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:57.611402 10916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:57.622437 10916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:57.624042 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:57.625597 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:57.639568 10923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:57.639537 10924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:57.639801 10926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:57.640717 10916 server_base.cc:1034] running on GCE node
I20260807 22:48:57.641588 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:57.643371 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:57.644613 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786142937644538 us; error 60 us; skew 500 ppm
I20260807 22:48:57.645126 10916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:57 dist-test-slave-csg5 krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142937, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:57.648954 10916 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:57.650748 10916 webserver.cc:460] Webserver started at http://127.0.0.1:37263/ using document root <none> and password file <none>
I20260807 22:48:57.651496 10916 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:57.651614 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:57.651916 10916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:57.655575 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "220be969b82a485a95d3791752d12505"
format_stamp: "Formatted at 2026-08-07 22:48:57 on dist-test-slave-csg5"
server_key: "f25a08f0adf74ce970341fde0cd9e11f"
server_key_iv: "ecf8b3fa912c39b3a241c8e46831debb"
server_key_version: "encryptionkey@0"
I20260807 22:48:57.656488 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "220be969b82a485a95d3791752d12505"
format_stamp: "Formatted at 2026-08-07 22:48:57 on dist-test-slave-csg5"
server_key: "f25a08f0adf74ce970341fde0cd9e11f"
server_key_iv: "ecf8b3fa912c39b3a241c8e46831debb"
server_key_version: "encryptionkey@0"
I20260807 22:48:57.662544 10916 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 22:48:57.666975 10933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:57.668772 10916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 22:48:57.669065 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "220be969b82a485a95d3791752d12505"
format_stamp: "Formatted at 2026-08-07 22:48:57 on dist-test-slave-csg5"
server_key: "f25a08f0adf74ce970341fde0cd9e11f"
server_key_iv: "ecf8b3fa912c39b3a241c8e46831debb"
server_key_version: "encryptionkey@0"
I20260807 22:48:57.669324 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:57.701828 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:57.707221 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:57.707628 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:48:57.731400 10916 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46249
I20260807 22:48:57.731549 10986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46249 every 8 connection(s)
I20260807 22:48:57.733150 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:48:57.735041 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 10916
I20260807 22:48:57.735252 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:48:57.735852 20522 external_mini_cluster.cc:1428] Setting key d87022da87dd66c35a1e35f426f3cb35
I20260807 22:48:57.738946 10987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:48:57.750867 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:57.738089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37755 (local address 127.0.0.1:46249)
0807 22:48:57.739253 (+ 1164us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:57.739268 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:57.739324 (+ 56us) server_negotiation.cc:407] Connection header received
0807 22:48:57.741132 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:57.741155 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:57.741598 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:57.742072 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:57.742764 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:57.744266 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:57.745722 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:57.746468 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:57.746703 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:57.748121 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:57.748129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:57.748146 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:57.748793 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:57.748882 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:57.748910 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:57.749068 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:57.749319 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:48:57.749680 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":465,"threads_started":1}
I20260807 22:48:57.752547 10987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505: Bootstrap starting.
I20260807 22:48:57.755862 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:48:57.756529 10987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505: Neither blocks nor log segments found. Creating new log.
I20260807 22:48:57.757728 10987 log.cc:824] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505: Log is configured to *not* fsync() on all Append() calls
I20260807 22:48:57.761566 10987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505: No bootstrap required, opened a new log
I20260807 22:48:57.768666 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:48:57.761565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57629 (local address 127.0.0.1:46249)
0807 22:48:57.761790 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:48:57.761798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:48:57.762192 (+ 394us) server_negotiation.cc:407] Connection header received
0807 22:48:57.762459 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:48:57.762464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:48:57.762538 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:48:57.762669 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:48:57.763344 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:57.764202 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:57.765741 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:48:57.766261 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:48:57.766439 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:48:57.767762 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:48:57.767767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:48:57.767774 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:48:57.767796 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:48:57.767840 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:48:57.767858 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:48:57.767996 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:48:57.768174 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:48:57.768287 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:48:57.771256 10987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260807 22:48:57.771735 10987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:48:57.771865 10987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220be969b82a485a95d3791752d12505, State: Initialized, Role: FOLLOWER
I20260807 22:48:57.772522 10987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260807 22:48:57.772826 10987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:48:57.773033 10987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:48:57.773240 10987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:48:57.776413 10987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260807 22:48:57.777009 10987 leader_election.cc:302] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 220be969b82a485a95d3791752d12505; no voters:
I20260807 22:48:57.777726 10987 leader_election.cc:288] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:48:57.777930 11000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:48:57.778929 11000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [term 1 LEADER]: Becoming Leader. State: Replica: 220be969b82a485a95d3791752d12505, State: Running, Role: LEADER
I20260807 22:48:57.779605 11000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260807 22:48:57.780318 10987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:48:57.782877 11003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "220be969b82a485a95d3791752d12505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } } }
I20260807 22:48:57.782989 11004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220be969b82a485a95d3791752d12505. Latest consensus state: current_term: 1 leader_uuid: "220be969b82a485a95d3791752d12505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220be969b82a485a95d3791752d12505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } } }
I20260807 22:48:57.783439 11003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:57.783562 11004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505 [sys.catalog]: This master's current role is: LEADER
I20260807 22:48:57.786108 11008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:48:57.791890 11008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:48:57.802613 11008 catalog_manager.cc:1347] Generated new cluster ID: b03b52b358d64566877e9de40df705e2
I20260807 22:48:57.802785 11008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:48:57.831383 11008 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:48:57.833637 11008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:48:57.850970 11008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 220be969b82a485a95d3791752d12505: Generated new TSK 0
I20260807 22:48:57.852079 11008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:48:57.873019 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10916
W20260807 22:48:57.874432 10982 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37755
W20260807 22:48:57.875545 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 10916: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:48:57.875725 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 10916
2026-08-07T22:48:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:48:58 dist-test-slave-csg5 krb5kdc[11024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:48:58 dist-test-slave-csg5 krb5kdc[11024](info): set up 2 sockets
Aug 07 22:48:58 dist-test-slave-csg5 krb5kdc[11024](info): commencing operation
krb5kdc: starting...
W20260807 22:48:59.104856 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:48:59 dist-test-slave-csg5 krb5kdc[11024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:48:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:48:59.507582 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:48:59.905272 11038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:48:59.905663 11038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:48:59.905856 11038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:48:59.915053 11038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:48:59.915211 11038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:48:59.915308 11038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:48:59.915412 11038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:48:59.915491 11038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:48:59.926625 11038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:48:59.928122 11038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:48:59.929622 11038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:48:59.943682 11044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:59.943681 11047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:48:59.943677 11045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:48:59.945355 11038 server_base.cc:1034] running on GCE node
I20260807 22:48:59.946390 11038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:48:59.948180 11038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:48:59.949506 11038 hybrid_clock.cc:648] HybridClock initialized: now 1786142939949463 us; error 72 us; skew 500 ppm
I20260807 22:48:59.949994 11038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:48:59 dist-test-slave-csg5 krb5kdc[11024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142939, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:48:59.958382 11038 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:48:59.960544 11038 webserver.cc:460] Webserver started at http://127.0.0.1:34375/ using document root <none> and password file <none>
I20260807 22:48:59.961400 11038 fs_manager.cc:359] Metadata directory not provided
I20260807 22:48:59.961585 11038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:48:59.961964 11038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:48:59.965940 11038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "ebea5d4954c14b2086b7c88cad57f722"
format_stamp: "Formatted at 2026-08-07 22:48:59 on dist-test-slave-csg5"
server_key: "19f7dd68faec157a8e5856b70ab8e4e2"
server_key_iv: "cf40e1f0ddca0a1e601668a7be3f9e30"
server_key_version: "encryptionkey@0"
I20260807 22:48:59.967002 11038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "ebea5d4954c14b2086b7c88cad57f722"
format_stamp: "Formatted at 2026-08-07 22:48:59 on dist-test-slave-csg5"
server_key: "19f7dd68faec157a8e5856b70ab8e4e2"
server_key_iv: "cf40e1f0ddca0a1e601668a7be3f9e30"
server_key_version: "encryptionkey@0"
I20260807 22:48:59.973105 11038 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260807 22:48:59.977502 11054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:48:59.979477 11038 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:48:59.979700 11038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "ebea5d4954c14b2086b7c88cad57f722"
format_stamp: "Formatted at 2026-08-07 22:48:59 on dist-test-slave-csg5"
server_key: "19f7dd68faec157a8e5856b70ab8e4e2"
server_key_iv: "cf40e1f0ddca0a1e601668a7be3f9e30"
server_key_version: "encryptionkey@0"
I20260807 22:48:59.979957 11038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:48:59.987152 11038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:48:59.992611 11038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:48:59.992995 11038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:00.017361 11038 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37445
I20260807 22:49:00.017478 11113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37445 every 8 connection(s)
I20260807 22:49:00.018996 11038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:00.023444 11114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:00.024442 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11038
I20260807 22:49:00.024677 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:00.025616 20522 external_mini_cluster.cc:1428] Setting key 33ddf742d0c63f50a4727c9d2092cec8
I20260807 22:49:00.036813 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:00.027870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37935 (local address 127.0.0.1:37445)
0807 22:49:00.028891 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:00.028905 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:00.028956 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:49:00.032386 (+ 3430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:00.032420 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:00.032904 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:00.033440 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:00.034762 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:00.034769 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:00.034784 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:00.034804 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:00.034877 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:00.034903 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:00.035395 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:00.035586 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 22:49:00.035782 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":481,"threads_started":1}
I20260807 22:49:00.041464 11114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722: Bootstrap starting.
I20260807 22:49:00.041791 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:00.045305 11114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:00.046495 11114 log.cc:824] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:00.050822 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:00.046724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58277 (local address 127.0.0.1:37445)
0807 22:49:00.047048 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:00.047057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:00.047635 (+ 578us) server_negotiation.cc:407] Connection header received
0807 22:49:00.047983 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:00.047992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:00.048071 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:00.048215 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:00.049638 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:00.049643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:00.049650 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:00.049669 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:00.049957 (+ 288us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:00.049993 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:00.050130 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:00.050402 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:49:00.050467 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260807 22:49:00.050981 11114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722: No bootstrap required, opened a new log
I20260807 22:49:00.059305 11114 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } }
I20260807 22:49:00.059737 11114 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:00.059827 11114 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebea5d4954c14b2086b7c88cad57f722, State: Initialized, Role: FOLLOWER
I20260807 22:49:00.060364 11114 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } }
I20260807 22:49:00.060585 11114 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:00.060691 11114 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:00.060824 11114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:00.063341 11114 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } }
I20260807 22:49:00.063861 11114 leader_election.cc:302] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebea5d4954c14b2086b7c88cad57f722; no voters:
I20260807 22:49:00.064621 11114 leader_election.cc:288] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:00.064846 11124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:00.066372 11124 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [term 1 LEADER]: Becoming Leader. State: Replica: ebea5d4954c14b2086b7c88cad57f722, State: Running, Role: LEADER
I20260807 22:49:00.067291 11114 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:00.067234 11124 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } }
I20260807 22:49:00.071113 11126 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebea5d4954c14b2086b7c88cad57f722. Latest consensus state: current_term: 1 leader_uuid: "ebea5d4954c14b2086b7c88cad57f722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } } }
I20260807 22:49:00.071630 11125 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebea5d4954c14b2086b7c88cad57f722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebea5d4954c14b2086b7c88cad57f722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37445 } } }
I20260807 22:49:00.071916 11126 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:00.071972 11125 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:00.076453 11134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:00.082258 11134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:00.094722 11134 catalog_manager.cc:1347] Generated new cluster ID: 47932f27a96c4220a6cccb0ebe5c15ab
I20260807 22:49:00.094961 11134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:00.113932 11134 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:00.114563 11134 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:00.126823 11134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebea5d4954c14b2086b7c88cad57f722: Generated new TSK 0
I20260807 22:49:00.127967 11134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:00.141511 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11038
W20260807 22:49:00.142895 11109 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37935
W20260807 22:49:00.143795 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11038: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:00.143967 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11038
2026-08-07T22:49:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:00 dist-test-slave-csg5 krb5kdc[11145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:00 dist-test-slave-csg5 krb5kdc[11145](info): set up 2 sockets
Aug 07 22:49:00 dist-test-slave-csg5 krb5kdc[11145](info): commencing operation
krb5kdc: starting...
W20260807 22:49:01.382171 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.004s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:01 dist-test-slave-csg5 krb5kdc[11145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:01.792071 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:02.178705 11159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:02.179154 11159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:02.179343 11159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:02.187546 11159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:02.187784 11159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:02.187903 11159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:02.188016 11159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:02.188095 11159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:02.199182 11159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:02.200812 11159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:02.202313 11159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:02.216001 11166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:02.216084 11167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:02.216897 11169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:02.217545 11159 server_base.cc:1034] running on GCE node
I20260807 22:49:02.218562 11159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:02.220367 11159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:02.221632 11159 hybrid_clock.cc:648] HybridClock initialized: now 1786142942221588 us; error 46 us; skew 500 ppm
I20260807 22:49:02.222152 11159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:02 dist-test-slave-csg5 krb5kdc[11145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142942, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:02.226313 11159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:02.229208 11159 webserver.cc:460] Webserver started at http://127.0.0.1:40597/ using document root <none> and password file <none>
I20260807 22:49:02.230124 11159 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:02.230264 11159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:02.230732 11159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:02.234647 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "bdcd04de89064e23906601270114ea21"
format_stamp: "Formatted at 2026-08-07 22:49:02 on dist-test-slave-csg5"
server_key: "6ba13b00a0024ddaa24f8cd7be05e1e6"
server_key_iv: "2a3219f8a34ddce30d80651461e33270"
server_key_version: "encryptionkey@0"
I20260807 22:49:02.235813 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "bdcd04de89064e23906601270114ea21"
format_stamp: "Formatted at 2026-08-07 22:49:02 on dist-test-slave-csg5"
server_key: "6ba13b00a0024ddaa24f8cd7be05e1e6"
server_key_iv: "2a3219f8a34ddce30d80651461e33270"
server_key_version: "encryptionkey@0"
I20260807 22:49:02.242564 11159 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 22:49:02.247740 11180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:02.250263 11159 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260807 22:49:02.250566 11159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "bdcd04de89064e23906601270114ea21"
format_stamp: "Formatted at 2026-08-07 22:49:02 on dist-test-slave-csg5"
server_key: "6ba13b00a0024ddaa24f8cd7be05e1e6"
server_key_iv: "2a3219f8a34ddce30d80651461e33270"
server_key_version: "encryptionkey@0"
I20260807 22:49:02.250959 11159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:02.280584 11159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:02.286207 11159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:02.286586 11159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:02.310914 11159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39873
I20260807 22:49:02.310961 11238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39873 every 8 connection(s)
I20260807 22:49:02.312644 11159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:02.313848 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11159
I20260807 22:49:02.314178 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:02.315312 20522 external_mini_cluster.cc:1428] Setting key 418b112a8a2867f08865a6fd942fcbcc
I20260807 22:49:02.318575 11239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 22:49:02 dist-test-slave-csg5 krb5kdc[11145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:49:02.333565 11239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21: Bootstrap starting.
I20260807 22:49:02.337234 11239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:02.338423 11239 log.cc:824] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:02.342587 11239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21: No bootstrap required, opened a new log
I20260807 22:49:02.343096 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:02.317536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:38003 (local address 127.0.0.1:39873)
0807 22:49:02.318513 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:02.318526 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:02.318577 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:49:02.320560 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:02.320590 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:02.321037 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:02.321442 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:02.323576 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:02.325260 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:02.327146 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:02.327988 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:02.328147 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:02.333849 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:02.333860 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:02.333877 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:02.333925 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:02.337346 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:02.338438 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:02.338446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:02.338453 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:02.338788 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:02.339573 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:02.339576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:02.339578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:02.340402 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:02.340760 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:02.341242 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 22:49:02.341619 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":417,"threads_started":1}
I20260807 22:49:02.348228 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:02.354918 11239 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } }
I20260807 22:49:02.355592 11239 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:02.355772 11239 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdcd04de89064e23906601270114ea21, State: Initialized, Role: FOLLOWER
I20260807 22:49:02.356832 11239 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } }
I20260807 22:49:02.357218 11239 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:02.357404 11239 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:02.357645 11239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:02.361037 11239 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } }
I20260807 22:49:02.361809 11239 leader_election.cc:302] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdcd04de89064e23906601270114ea21; no voters:
I20260807 22:49:02.362812 11239 leader_election.cc:288] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:02.362975 11249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:02.364259 11249 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [term 1 LEADER]: Becoming Leader. State: Replica: bdcd04de89064e23906601270114ea21, State: Running, Role: LEADER
I20260807 22:49:02.365022 11249 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } }
I20260807 22:49:02.365254 11239 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:02.368162 11251 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdcd04de89064e23906601270114ea21. Latest consensus state: current_term: 1 leader_uuid: "bdcd04de89064e23906601270114ea21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } } }
I20260807 22:49:02.369105 11251 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:02.369180 11250 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdcd04de89064e23906601270114ea21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdcd04de89064e23906601270114ea21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39873 } } }
I20260807 22:49:02.369760 11250 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:02.383811 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:02.356482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:37651 (local address 127.0.0.1:39873)
0807 22:49:02.356674 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:02.356681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:02.359384 (+ 2703us) server_negotiation.cc:407] Connection header received
0807 22:49:02.359669 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:02.359677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:02.359784 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:02.359875 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:02.362449 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:02.364204 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:02.366307 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:02.367250 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:02.367403 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:02.371944 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:02.371954 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:02.371962 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:02.372052 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:02.377356 (+ 5304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:02.379072 (+ 1716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:02.379090 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:02.379096 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:02.379325 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:02.380178 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:02.380187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:02.380193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:02.380753 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:02.381170 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:02.382834 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0807 22:49:02.383176 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:49:02.387360 11255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:02.394032 11255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:02.407292 11255 catalog_manager.cc:1347] Generated new cluster ID: 23f924a9a7844566810804b8cf3d3c01
I20260807 22:49:02.407439 11255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:02.431459 11255 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:02.433076 11255 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:02.448040 11255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdcd04de89064e23906601270114ea21: Generated new TSK 0
I20260807 22:49:02.449527 11255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:02.463739 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11159
I20260807 22:49:02.465663 11233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:02.570390 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11159
W20260807 22:49:02.594942 11143 connection.cc:535] client connection to 127.0.0.1:39873 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39873): Connection reset by peer (error 104)
2026-08-07T22:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:02 dist-test-slave-csg5 krb5kdc[11271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:02 dist-test-slave-csg5 krb5kdc[11271](info): set up 2 sockets
Aug 07 22:49:02 dist-test-slave-csg5 krb5kdc[11271](info): commencing operation
krb5kdc: starting...
W20260807 22:49:03.825963 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:04 dist-test-slave-csg5 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:04.223385 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:04.601540 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:04.601861 11286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:04.601958 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:04.609426 11286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:04.609563 11286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:04.609611 11286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:04.609685 11286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:04.609732 11286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:04.620661 11286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:04.622141 11286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:04.623558 11286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:04.637089 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:04.637106 11295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:04.637285 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:04.638577 11286 server_base.cc:1034] running on GCE node
I20260807 22:49:04.639668 11286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:04.641605 11286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:04.642839 11286 hybrid_clock.cc:648] HybridClock initialized: now 1786142944642787 us; error 39 us; skew 500 ppm
I20260807 22:49:04.643314 11286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:04 dist-test-slave-csg5 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142944, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:04.647411 11286 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:04.649224 11286 webserver.cc:460] Webserver started at http://127.0.0.1:44171/ using document root <none> and password file <none>
I20260807 22:49:04.649988 11286 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:04.650156 11286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:04.650490 11286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:04.653887 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "4e3c6edb538b400289c22ad13b6ee913"
format_stamp: "Formatted at 2026-08-07 22:49:04 on dist-test-slave-csg5"
server_key: "6dc51f9a715a42a6ebbf34a4309d7a4d"
server_key_iv: "54c45c3aa11373f555df9f19487a5c22"
server_key_version: "encryptionkey@0"
I20260807 22:49:04.654875 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "4e3c6edb538b400289c22ad13b6ee913"
format_stamp: "Formatted at 2026-08-07 22:49:04 on dist-test-slave-csg5"
server_key: "6dc51f9a715a42a6ebbf34a4309d7a4d"
server_key_iv: "54c45c3aa11373f555df9f19487a5c22"
server_key_version: "encryptionkey@0"
I20260807 22:49:04.661082 11286 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:49:04.665275 11302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:04.667057 11286 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:49:04.667284 11286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "4e3c6edb538b400289c22ad13b6ee913"
format_stamp: "Formatted at 2026-08-07 22:49:04 on dist-test-slave-csg5"
server_key: "6dc51f9a715a42a6ebbf34a4309d7a4d"
server_key_iv: "54c45c3aa11373f555df9f19487a5c22"
server_key_version: "encryptionkey@0"
I20260807 22:49:04.667516 11286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:04.690956 11286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:04.696094 11286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:04.696439 11286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:04.719224 11286 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36937
I20260807 22:49:04.719269 11364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36937 every 8 connection(s)
I20260807 22:49:04.720801 11286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:04.722703 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11286
I20260807 22:49:04.722909 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:04.723526 20522 external_mini_cluster.cc:1428] Setting key 47ef35b05b70688cc1951e8e1ab75067
I20260807 22:49:04.726089 11365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:04.739835 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:04.725607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:33807 (local address 127.0.0.1:36937)
0807 22:49:04.726607 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:04.726620 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:04.726721 (+ 101us) server_negotiation.cc:407] Connection header received
0807 22:49:04.728499 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:04.728523 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:04.729076 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:04.729481 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:04.730248 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:04.731809 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:04.734123 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:04.735043 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:04.735208 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:04.736876 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:04.736885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:04.736901 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:04.737623 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:04.737764 (+ 141us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:04.737796 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:04.737943 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:04.738402 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:49:04.738633 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":481,"threads_started":1}
I20260807 22:49:04.740321 11365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913: Bootstrap starting.
I20260807 22:49:04.744349 11365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:04.744742 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:04.745851 11365 log.cc:824] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:04.749786 11365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913: No bootstrap required, opened a new log
I20260807 22:49:04.758158 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:04.750016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:40779 (local address 127.0.0.1:36937)
0807 22:49:04.750218 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:04.750226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:04.750814 (+ 588us) server_negotiation.cc:407] Connection header received
0807 22:49:04.751019 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:04.751025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:04.751121 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:04.751256 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:04.751840 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:04.753226 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:04.754752 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:04.755600 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:04.755756 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:04.756992 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:04.756999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:04.757005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:04.757042 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:04.757119 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:04.757144 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:04.757282 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:04.757683 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 22:49:04.757749 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:49:04.761104 11365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260807 22:49:04.761590 11365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:04.761724 11365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3c6edb538b400289c22ad13b6ee913, State: Initialized, Role: FOLLOWER
I20260807 22:49:04.762392 11365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260807 22:49:04.762712 11365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:04.762907 11365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:04.763091 11365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:04.766039 11365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260807 22:49:04.766767 11365 leader_election.cc:302] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e3c6edb538b400289c22ad13b6ee913; no voters:
I20260807 22:49:04.767436 11365 leader_election.cc:288] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:04.767623 11375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:04.768437 11375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [term 1 LEADER]: Becoming Leader. State: Replica: 4e3c6edb538b400289c22ad13b6ee913, State: Running, Role: LEADER
I20260807 22:49:04.769042 11375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260807 22:49:04.769868 11365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:04.772518 11377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e3c6edb538b400289c22ad13b6ee913. Latest consensus state: current_term: 1 leader_uuid: "4e3c6edb538b400289c22ad13b6ee913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260807 22:49:04.772962 11377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:04.774338 11376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e3c6edb538b400289c22ad13b6ee913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3c6edb538b400289c22ad13b6ee913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260807 22:49:04.774799 11376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:04.776011 11384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:04.782220 11384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:04.792452 11384 catalog_manager.cc:1347] Generated new cluster ID: a76d2553530b4daca87f38544f4843cd
I20260807 22:49:04.792605 11384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:04.817241 11384 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:04.818753 11384 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:04.834270 11384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e3c6edb538b400289c22ad13b6ee913: Generated new TSK 0
I20260807 22:49:04.835248 11384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:04.847779 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11286
W20260807 22:49:04.849298 11358 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.95:33807
W20260807 22:49:04.850165 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11286: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:04.850426 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11286
W20260807 22:49:04.872977 11269 connection.cc:535] client connection to 127.0.0.1:36937 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36937): Connection reset by peer (error 104)
2026-08-07T22:49:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:04 dist-test-slave-csg5 krb5kdc[11398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:04 dist-test-slave-csg5 krb5kdc[11398](info): set up 2 sockets
Aug 07 22:49:04 dist-test-slave-csg5 krb5kdc[11398](info): commencing operation
krb5kdc: starting...
W20260807 22:49:06.159960 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:06 dist-test-slave-csg5 krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:06.567750 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:06.938565 11412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:06.938977 11412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:06.939083 11412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:06.946811 11412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:06.946946 11412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:06.946997 11412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:06.947062 11412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:06.947109 11412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:06.957897 11412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:06.959265 11412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:06.960598 11412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:06.973770 11419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:06.973961 11418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:06.973973 11421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:06.975081 11412 server_base.cc:1034] running on GCE node
I20260807 22:49:06.976018 11412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:06.977625 11412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:06.978874 11412 hybrid_clock.cc:648] HybridClock initialized: now 1786142946978796 us; error 81 us; skew 500 ppm
I20260807 22:49:06.979267 11412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:06 dist-test-slave-csg5 krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:06.983119 11412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:06.984923 11412 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
I20260807 22:49:06.985692 11412 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:06.985858 11412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:06.986194 11412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:06.989610 11412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "aa0e161faa624d91ab176cf47bb1fa52"
format_stamp: "Formatted at 2026-08-07 22:49:06 on dist-test-slave-csg5"
server_key: "21e8a50d6f7d2e6724dfdf78ad10c3df"
server_key_iv: "d0829b7973053d8b29e395b64c9459f0"
server_key_version: "encryptionkey@0"
I20260807 22:49:06.990481 11412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "aa0e161faa624d91ab176cf47bb1fa52"
format_stamp: "Formatted at 2026-08-07 22:49:06 on dist-test-slave-csg5"
server_key: "21e8a50d6f7d2e6724dfdf78ad10c3df"
server_key_iv: "d0829b7973053d8b29e395b64c9459f0"
server_key_version: "encryptionkey@0"
I20260807 22:49:06.996536 11412 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:49:07.000679 11428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:07.002333 11412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:49:07.002583 11412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "aa0e161faa624d91ab176cf47bb1fa52"
format_stamp: "Formatted at 2026-08-07 22:49:06 on dist-test-slave-csg5"
server_key: "21e8a50d6f7d2e6724dfdf78ad10c3df"
server_key_iv: "d0829b7973053d8b29e395b64c9459f0"
server_key_version: "encryptionkey@0"
I20260807 22:49:07.002898 11412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:07.009745 11412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:07.014948 11412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:07.015324 11412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:07.038718 11412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36967
I20260807 22:49:07.038823 11485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36967 every 8 connection(s)
I20260807 22:49:07.040460 11412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:07.044853 11486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:07.049746 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11412
I20260807 22:49:07.049978 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:07.050551 20522 external_mini_cluster.cc:1428] Setting key 0bc28f274557044d0ef5f552873ae9f5
I20260807 22:49:07.057621 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52: Bootstrap starting.
I20260807 22:49:07.059625 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:07.052733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:50049 (local address 127.0.0.1:36967)
0807 22:49:07.053634 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:07.053646 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:07.053692 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:49:07.055937 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:07.055961 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:07.056336 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:07.056676 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:07.057912 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:07.057918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:07.057935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:07.057951 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:07.058014 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:07.058035 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:07.058228 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:07.058344 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 22:49:07.058558 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":436,"threads_started":1}
I20260807 22:49:07.061303 11486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:07.062638 11486 log.cc:824] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:07.063836 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:07.066542 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52: No bootstrap required, opened a new log
I20260807 22:49:07.072914 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:07.069802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:46123 (local address 127.0.0.1:36967)
0807 22:49:07.070065 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:07.070072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:07.070498 (+ 426us) server_negotiation.cc:407] Connection header received
0807 22:49:07.070684 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:07.070690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:07.070769 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:07.070897 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:07.072032 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:07.072038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:07.072043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:07.072063 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:07.072124 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:07.072152 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:07.072297 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:07.072458 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 22:49:07.072530 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:49:07.076768 11486 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 22:49:07.077227 11486 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:07.077318 11486 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e161faa624d91ab176cf47bb1fa52, State: Initialized, Role: FOLLOWER
I20260807 22:49:07.077900 11486 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 22:49:07.078177 11486 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:07.078341 11486 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:07.078543 11486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:07.081189 11486 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 22:49:07.081802 11486 leader_election.cc:302] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa0e161faa624d91ab176cf47bb1fa52; no voters:
I20260807 22:49:07.082448 11486 leader_election.cc:288] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:07.082700 11498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:07.083958 11498 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [term 1 LEADER]: Becoming Leader. State: Replica: aa0e161faa624d91ab176cf47bb1fa52, State: Running, Role: LEADER
I20260807 22:49:07.084574 11498 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 22:49:07.084877 11486 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:07.087579 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa0e161faa624d91ab176cf47bb1fa52. Latest consensus state: current_term: 1 leader_uuid: "aa0e161faa624d91ab176cf47bb1fa52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260807 22:49:07.087961 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:07.088500 11499 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa0e161faa624d91ab176cf47bb1fa52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0e161faa624d91ab176cf47bb1fa52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260807 22:49:07.088882 11499 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:07.088996 11506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:07.095304 11506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:07.106338 11506 catalog_manager.cc:1347] Generated new cluster ID: cfc2cb17a7cb4dceb1177a69da882a3d
I20260807 22:49:07.106473 11506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:07.126977 11506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:07.127527 11506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:07.141374 11506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa0e161faa624d91ab176cf47bb1fa52: Generated new TSK 0
I20260807 22:49:07.142313 11506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:07.154727 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11412
W20260807 22:49:07.156189 11481 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.95:50049
W20260807 22:49:07.157007 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:07.157161 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11412
2026-08-07T22:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:07 dist-test-slave-csg5 krb5kdc[11526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:07 dist-test-slave-csg5 krb5kdc[11526](info): set up 2 sockets
Aug 07 22:49:07 dist-test-slave-csg5 krb5kdc[11526](info): commencing operation
krb5kdc: starting...
W20260807 22:49:08.403524 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.010s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:08 dist-test-slave-csg5 krb5kdc[11526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:08.804294 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:09.199224 11539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:09.199548 11539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:09.199743 11539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:09.207511 11539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:09.207654 11539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:09.207749 11539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:09.207836 11539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:09.207907 11539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:09.218591 11539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:09.220162 11539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:09.221585 11539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:09.234874 11546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:09.234846 11550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:09.234822 11547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:09.235781 11539 server_base.cc:1034] running on GCE node
I20260807 22:49:09.236729 11539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:09.238385 11539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:09.239682 11539 hybrid_clock.cc:648] HybridClock initialized: now 1786142949239600 us; error 74 us; skew 500 ppm
I20260807 22:49:09.240149 11539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:09 dist-test-slave-csg5 krb5kdc[11526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:09.243917 11539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:09.245726 11539 webserver.cc:460] Webserver started at http://127.0.0.1:33943/ using document root <none> and password file <none>
I20260807 22:49:09.246481 11539 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:09.246642 11539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:09.247020 11539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:09.250474 11539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "1c2d37d1e1a94d2eae59b2c24755ea17"
format_stamp: "Formatted at 2026-08-07 22:49:09 on dist-test-slave-csg5"
server_key: "29f5119c63f36e3401fa58b0101264be"
server_key_iv: "557af94c79526f33104656db06569035"
server_key_version: "encryptionkey@0"
I20260807 22:49:09.251492 11539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "1c2d37d1e1a94d2eae59b2c24755ea17"
format_stamp: "Formatted at 2026-08-07 22:49:09 on dist-test-slave-csg5"
server_key: "29f5119c63f36e3401fa58b0101264be"
server_key_iv: "557af94c79526f33104656db06569035"
server_key_version: "encryptionkey@0"
I20260807 22:49:09.257629 11539 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 22:49:09.261691 11557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:09.263557 11539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:49:09.263783 11539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "1c2d37d1e1a94d2eae59b2c24755ea17"
format_stamp: "Formatted at 2026-08-07 22:49:09 on dist-test-slave-csg5"
server_key: "29f5119c63f36e3401fa58b0101264be"
server_key_iv: "557af94c79526f33104656db06569035"
server_key_version: "encryptionkey@0"
I20260807 22:49:09.264017 11539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:09.290181 11539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:09.295368 11539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:09.295717 11539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:09.319610 11539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39803
I20260807 22:49:09.319676 11612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39803 every 8 connection(s)
I20260807 22:49:09.321214 11539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:09.325048 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11539
I20260807 22:49:09.325270 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:09.325748 20522 external_mini_cluster.cc:1428] Setting key 03df3bb649d9441e2bd0729a3a384e94
I20260807 22:49:09.325796 11613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:09.342276 11613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17: Bootstrap starting.
Aug 07 22:49:09 dist-test-slave-csg5 krb5kdc[11526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:49:09.346412 11613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:09.347760 11613 log.cc:824] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:09.351920 11613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17: No bootstrap required, opened a new log
I20260807 22:49:09.354328 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:09.327706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:37345 (local address 127.0.0.1:39803)
0807 22:49:09.328634 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:09.328646 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:09.328698 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:49:09.331313 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:09.331341 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:09.331855 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:09.332253 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:09.334358 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:09.335844 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:09.337728 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:09.338789 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:09.338970 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:09.344735 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:09.344748 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:09.344764 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:09.344822 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:09.348904 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:09.350383 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:09.350392 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:09.350399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:09.350590 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:09.351348 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:09.351352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:09.351354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:09.351939 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:09.352316 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:09.352857 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 22:49:09.353212 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":442,"threads_started":1}
I20260807 22:49:09.358845 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:09.361512 11613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } }
I20260807 22:49:09.362107 11613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:09.362253 11613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2d37d1e1a94d2eae59b2c24755ea17, State: Initialized, Role: FOLLOWER
I20260807 22:49:09.363030 11613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } }
I20260807 22:49:09.363327 11613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:09.363498 11613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:09.363704 11613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:09.367859 11613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } }
I20260807 22:49:09.368587 11613 leader_election.cc:302] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2d37d1e1a94d2eae59b2c24755ea17; no voters:
I20260807 22:49:09.369490 11613 leader_election.cc:288] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:09.369664 11625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:09.371133 11625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [term 1 LEADER]: Becoming Leader. State: Replica: 1c2d37d1e1a94d2eae59b2c24755ea17, State: Running, Role: LEADER
I20260807 22:49:09.371985 11625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } }
I20260807 22:49:09.372510 11613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:09.380016 11626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } } }
I20260807 22:49:09.380517 11626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:09.386951 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:09.364102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:44877 (local address 127.0.0.1:39803)
0807 22:49:09.364353 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:09.364364 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:09.365024 (+ 660us) server_negotiation.cc:407] Connection header received
0807 22:49:09.365250 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:09.365255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:09.365335 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:09.365443 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:09.367734 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:09.368766 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:09.370876 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:09.371703 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:09.371879 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:09.375646 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:09.375657 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:09.375663 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:09.375717 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:09.380040 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:09.381801 (+ 1761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:09.381811 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:09.381818 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:09.382017 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:09.382990 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:09.382999 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:09.383006 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:09.383642 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:09.384063 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:09.385953 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0807 22:49:09.386296 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:49:09.392381 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c2d37d1e1a94d2eae59b2c24755ea17. Latest consensus state: current_term: 1 leader_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2d37d1e1a94d2eae59b2c24755ea17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39803 } } }
I20260807 22:49:09.392763 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:09.393443 11634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:09.400518 11634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:09.413126 11634 catalog_manager.cc:1347] Generated new cluster ID: baebece167704442bed859566572ade4
I20260807 22:49:09.413444 11634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:09.483290 11634 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:09.484786 11634 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:09.493780 11634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c2d37d1e1a94d2eae59b2c24755ea17: Generated new TSK 0
I20260807 22:49:09.494917 11634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:09.505179 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11539
I20260807 22:49:09.507021 11608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:09.615724 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11539
W20260807 22:49:09.641053 11522 connection.cc:535] client connection to 127.0.0.1:39803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39803): Connection reset by peer (error 104)
2026-08-07T22:49:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:09 dist-test-slave-csg5 krb5kdc[11649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:09 dist-test-slave-csg5 krb5kdc[11649](info): set up 2 sockets
Aug 07 22:49:09 dist-test-slave-csg5 krb5kdc[11649](info): commencing operation
krb5kdc: starting...
W20260807 22:49:10.909647 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.006s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:11 dist-test-slave-csg5 krb5kdc[11649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:11.313133 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:11.689452 11665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:11.689832 11665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:11.689935 11665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:11.698498 11665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:11.698760 11665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:11.698850 11665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:11.698921 11665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:11.698969 11665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:11.712457 11665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:11.714396 11665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:11.715948 11665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:11.730060 11670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:11.730459 11671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:11.730692 11674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:11.731324 11665 server_base.cc:1034] running on GCE node
I20260807 22:49:11.732475 11665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:11.734292 11665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:11.735579 11665 hybrid_clock.cc:648] HybridClock initialized: now 1786142951735528 us; error 50 us; skew 500 ppm
I20260807 22:49:11.736086 11665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:11 dist-test-slave-csg5 krb5kdc[11649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:11.743789 11665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:11.745515 11665 webserver.cc:460] Webserver started at http://127.0.0.1:43777/ using document root <none> and password file <none>
I20260807 22:49:11.746244 11665 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:11.746362 11665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:11.746634 11665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:11.749955 11665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "821680e8e75448b3be54b0aee24aca07"
format_stamp: "Formatted at 2026-08-07 22:49:11 on dist-test-slave-csg5"
server_key: "91ef27db32670508ace2b475f18e51ca"
server_key_iv: "a92a6c3f3861f15107d63b3cabac3f83"
server_key_version: "encryptionkey@0"
I20260807 22:49:11.750942 11665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "821680e8e75448b3be54b0aee24aca07"
format_stamp: "Formatted at 2026-08-07 22:49:11 on dist-test-slave-csg5"
server_key: "91ef27db32670508ace2b475f18e51ca"
server_key_iv: "a92a6c3f3861f15107d63b3cabac3f83"
server_key_version: "encryptionkey@0"
I20260807 22:49:11.757073 11665 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 22:49:11.761439 11684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:11.763217 11665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:49:11.763427 11665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "821680e8e75448b3be54b0aee24aca07"
format_stamp: "Formatted at 2026-08-07 22:49:11 on dist-test-slave-csg5"
server_key: "91ef27db32670508ace2b475f18e51ca"
server_key_iv: "a92a6c3f3861f15107d63b3cabac3f83"
server_key_version: "encryptionkey@0"
I20260807 22:49:11.763638 11665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:11.783751 11665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:11.789134 11665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:11.789494 11665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:11.812453 11665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35419
I20260807 22:49:11.812503 11737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35419 every 8 connection(s)
I20260807 22:49:11.814056 11665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:11.818159 11738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:11.823938 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11665
I20260807 22:49:11.824173 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:11.824697 20522 external_mini_cluster.cc:1428] Setting key bbc50df1184d2f2286c89e5fdba47be0
I20260807 22:49:11.830400 11738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07: Bootstrap starting.
I20260807 22:49:11.834438 11738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:11.835805 11738 log.cc:824] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:11.839562 11738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07: No bootstrap required, opened a new log
I20260807 22:49:11.841071 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:11.826952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:49341 (local address 127.0.0.1:35419)
0807 22:49:11.827934 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:11.827947 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:11.828005 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:49:11.830000 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:11.830029 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:11.830535 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:11.831020 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:11.831744 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:11.833523 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:11.835559 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:11.836747 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:11.836990 (+ 243us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:11.838447 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:11.838455 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:11.838472 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:11.839141 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:11.839225 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:11.839253 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:11.839383 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:11.839757 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 22:49:11.839936 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":451,"threads_started":1}
I20260807 22:49:11.845822 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:11.848356 11738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } }
I20260807 22:49:11.848835 11738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:11.848973 11738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821680e8e75448b3be54b0aee24aca07, State: Initialized, Role: FOLLOWER
I20260807 22:49:11.849645 11738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } }
I20260807 22:49:11.850009 11738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:11.850167 11738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:11.850363 11738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:11.853798 11738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } }
I20260807 22:49:11.854450 11738 leader_election.cc:302] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 821680e8e75448b3be54b0aee24aca07; no voters:
I20260807 22:49:11.855302 11738 leader_election.cc:288] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:11.855422 11748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:11.856815 11748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [term 1 LEADER]: Becoming Leader. State: Replica: 821680e8e75448b3be54b0aee24aca07, State: Running, Role: LEADER
I20260807 22:49:11.857544 11748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } }
I20260807 22:49:11.858189 11738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:11.860666 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:11.851178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:45223 (local address 127.0.0.1:35419)
0807 22:49:11.851377 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:11.851382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:11.851882 (+ 500us) server_negotiation.cc:407] Connection header received
0807 22:49:11.852125 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:11.852135 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:11.852307 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:11.852445 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:11.853413 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:11.854465 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:11.856532 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:11.857354 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:11.857504 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:11.859050 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:11.859056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:11.859059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:11.859081 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:11.859140 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:11.859158 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:11.859327 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:11.860153 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 22:49:11.860223 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:49:11.860924 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "821680e8e75448b3be54b0aee24aca07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } } }
I20260807 22:49:11.862288 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:11.861860 11749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 821680e8e75448b3be54b0aee24aca07. Latest consensus state: current_term: 1 leader_uuid: "821680e8e75448b3be54b0aee24aca07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821680e8e75448b3be54b0aee24aca07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35419 } } }
I20260807 22:49:11.862998 11749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:11.869004 11753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:11.874799 11753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:11.884903 11753 catalog_manager.cc:1347] Generated new cluster ID: cfc46344791940bf9458876c7055edfc
I20260807 22:49:11.885052 11753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:11.907377 11753 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:11.909090 11753 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:11.917591 11753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 821680e8e75448b3be54b0aee24aca07: Generated new TSK 0
I20260807 22:49:11.918581 11753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:11.929554 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11665
W20260807 22:49:11.931365 11732 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.95:49341
W20260807 22:49:11.932348 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:11.932554 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11665
W20260807 22:49:11.956369 11646 connection.cc:535] client connection to 127.0.0.1:35419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35419): Connection reset by peer (error 104)
2026-08-07T22:49:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:12 dist-test-slave-csg5 krb5kdc[11773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:12 dist-test-slave-csg5 krb5kdc[11773](info): set up 2 sockets
Aug 07 22:49:12 dist-test-slave-csg5 krb5kdc[11773](info): commencing operation
krb5kdc: starting...
W20260807 22:49:13.181630 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:13 dist-test-slave-csg5 krb5kdc[11773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:49:13.587106 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:13.955320 11788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:13.955657 11788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:13.955760 11788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:13.963389 11788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:13.963546 11788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:13.963626 11788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:13.963688 11788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:13.963737 11788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:13.974198 11788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:13.975615 11788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:13.977181 11788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:13.990820 11797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:13.990753 11793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:13.990885 11794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:13.991711 11788 server_base.cc:1034] running on GCE node
I20260807 22:49:13.992719 11788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:13.994413 11788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:13.995728 11788 hybrid_clock.cc:648] HybridClock initialized: now 1786142953995698 us; error 39 us; skew 500 ppm
I20260807 22:49:13.996165 11788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:13 dist-test-slave-csg5 krb5kdc[11773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:13.999706 11788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:49:14.001528 11788 webserver.cc:460] Webserver started at http://127.0.0.1:44237/ using document root <none> and password file <none>
I20260807 22:49:14.002269 11788 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:14.002426 11788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:14.002846 11788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:14.006138 11788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8"
format_stamp: "Formatted at 2026-08-07 22:49:13 on dist-test-slave-csg5"
server_key: "801a1a198334147282c4cd89840bc6b9"
server_key_iv: "cd64e8428f5ce871f79441a394414538"
server_key_version: "encryptionkey@0"
I20260807 22:49:14.007071 11788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8"
format_stamp: "Formatted at 2026-08-07 22:49:13 on dist-test-slave-csg5"
server_key: "801a1a198334147282c4cd89840bc6b9"
server_key_iv: "cd64e8428f5ce871f79441a394414538"
server_key_version: "encryptionkey@0"
I20260807 22:49:14.013027 11788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 22:49:14.017609 11804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:14.019392 11788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:49:14.019618 11788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8"
format_stamp: "Formatted at 2026-08-07 22:49:13 on dist-test-slave-csg5"
server_key: "801a1a198334147282c4cd89840bc6b9"
server_key_iv: "cd64e8428f5ce871f79441a394414538"
server_key_version: "encryptionkey@0"
I20260807 22:49:14.019865 11788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:14.026854 11788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:14.032577 11788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:14.033021 11788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:14.054766 11788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34523
I20260807 22:49:14.054941 11860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34523 every 8 connection(s)
I20260807 22:49:14.056375 11788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:14.057852 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11788
I20260807 22:49:14.058059 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:14.058636 20522 external_mini_cluster.cc:1428] Setting key aa303033a91e3e58a8eee7a3ae21ec93
I20260807 22:49:14.061657 11861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:14.068296 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:14.060841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:54607 (local address 127.0.0.1:34523)
0807 22:49:14.061927 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:14.061941 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:14.061992 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:49:14.063933 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:14.063957 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:14.064367 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:14.064861 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:14.066154 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:14.066161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:14.066185 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:14.066203 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:14.066283 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:14.066312 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:14.066600 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:14.066846 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 22:49:14.067243 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":469,"threads_started":1}
I20260807 22:49:14.072582 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:14.074193 11861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8: Bootstrap starting.
I20260807 22:49:14.078316 11861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:14.079947 11861 log.cc:824] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:14.081287 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:14.077873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:43633 (local address 127.0.0.1:34523)
0807 22:49:14.078124 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:14.078139 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:14.078548 (+ 409us) server_negotiation.cc:407] Connection header received
0807 22:49:14.078859 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:14.078866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:14.078955 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:14.079087 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:14.080350 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:14.080356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:14.080360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:14.080380 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:14.080433 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:14.080454 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:14.080616 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:14.080839 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 22:49:14.080914 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 22:49:14.083938 11861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8: No bootstrap required, opened a new log
I20260807 22:49:14.092207 11861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260807 22:49:14.092658 11861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:14.092788 11861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 742d1d0ead2044fb9f6a9dbcc0f424d8, State: Initialized, Role: FOLLOWER
I20260807 22:49:14.093427 11861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260807 22:49:14.093643 11861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:14.093828 11861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:14.094050 11861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:14.096614 11861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260807 22:49:14.097162 11861 leader_election.cc:302] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 742d1d0ead2044fb9f6a9dbcc0f424d8; no voters:
I20260807 22:49:14.097803 11861 leader_election.cc:288] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:14.098013 11872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:14.098838 11872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [term 1 LEADER]: Becoming Leader. State: Replica: 742d1d0ead2044fb9f6a9dbcc0f424d8, State: Running, Role: LEADER
I20260807 22:49:14.099536 11872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260807 22:49:14.100183 11861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:14.104341 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } } }
I20260807 22:49:14.104496 11874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 742d1d0ead2044fb9f6a9dbcc0f424d8. Latest consensus state: current_term: 1 leader_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d1d0ead2044fb9f6a9dbcc0f424d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } } }
I20260807 22:49:14.104836 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:14.104979 11874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:14.106792 11882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:14.112106 11882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:14.122268 11882 catalog_manager.cc:1347] Generated new cluster ID: a47f96f7be0a40c084d44e6e8f5109b2
I20260807 22:49:14.122424 11882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:14.149832 11882 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:14.150527 11882 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:14.164572 11882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 742d1d0ead2044fb9f6a9dbcc0f424d8: Generated new TSK 0
I20260807 22:49:14.165887 11882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:14.177589 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11788
W20260807 22:49:14.178903 11856 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.95:54607
W20260807 22:49:14.179811 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11788: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:14.180003 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11788
2026-08-07T22:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2248 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:14 dist-test-slave-csg5 krb5kdc[11896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:14 dist-test-slave-csg5 krb5kdc[11896](info): set up 2 sockets
Aug 07 22:49:14 dist-test-slave-csg5 krb5kdc[11896](info): commencing operation
krb5kdc: starting...
W20260807 22:49:15.429018 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:15 dist-test-slave-csg5 krb5kdc[11896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:15.829990 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36221
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:38905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:36221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:16.208438 11910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:16.208824 11910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:16.209046 11910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:16.217725 11910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:16.217885 11910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:16.218003 11910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:16.218127 11910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:16.218222 11910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:16.229449 11910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:38905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:36221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:36221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:16.230988 11910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:16.232512 11910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:16.245710 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:16.245826 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:16.245790 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:16.247166 11910 server_base.cc:1034] running on GCE node
I20260807 22:49:16.248164 11910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:16.249956 11910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:16.251405 11910 hybrid_clock.cc:648] HybridClock initialized: now 1786142956251382 us; error 94 us; skew 500 ppm
I20260807 22:49:16.251899 11910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:16 dist-test-slave-csg5 krb5kdc[11896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:16.256541 11910 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:16.258785 11910 webserver.cc:460] Webserver started at http://127.20.10.190:45163/ using document root <none> and password file <none>
I20260807 22:49:16.259696 11910 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:16.259883 11910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:16.260277 11910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:16.263969 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "2726ea1269474eb2910acd8e0e3c294b"
format_stamp: "Formatted at 2026-08-07 22:49:16 on dist-test-slave-csg5"
server_key: "4cfda156266c69d252443f51d64fe604"
server_key_iv: "feb71f3c085644bf231847b05e15aec5"
server_key_version: "encryptionkey@0"
I20260807 22:49:16.264928 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "2726ea1269474eb2910acd8e0e3c294b"
format_stamp: "Formatted at 2026-08-07 22:49:16 on dist-test-slave-csg5"
server_key: "4cfda156266c69d252443f51d64fe604"
server_key_iv: "feb71f3c085644bf231847b05e15aec5"
server_key_version: "encryptionkey@0"
I20260807 22:49:16.271471 11910 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 22:49:16.276062 11926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:16.278167 11910 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:49:16.278450 11910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "2726ea1269474eb2910acd8e0e3c294b"
format_stamp: "Formatted at 2026-08-07 22:49:16 on dist-test-slave-csg5"
server_key: "4cfda156266c69d252443f51d64fe604"
server_key_iv: "feb71f3c085644bf231847b05e15aec5"
server_key_version: "encryptionkey@0"
I20260807 22:49:16.278782 11910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:16.297307 11910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:16.302388 11910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:16.302814 11910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:16.324534 11910 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:36221
I20260807 22:49:16.324568 11982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:36221 every 8 connection(s)
I20260807 22:49:16.326093 11910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:16.329447 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 11910
I20260807 22:49:16.329669 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:16.330188 20522 external_mini_cluster.cc:1428] Setting key 66d78b7c0c4643f8786e157bfc65cc2e
I20260807 22:49:16.330523 11983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:16.345863 11983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b: Bootstrap starting.
Aug 07 22:49:16 dist-test-slave-csg5 krb5kdc[11896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:16.350772 11983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:16.352556 11983 log.cc:824] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:16.357144 11983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b: No bootstrap required, opened a new log
I20260807 22:49:16.359324 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:16.332165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40997 (local address 127.20.10.190:36221)
0807 22:49:16.333146 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:16.333160 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:16.333208 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:49:16.336304 (+ 3096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:16.336332 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:16.336837 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:16.337311 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:16.338927 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:16.340241 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:16.341580 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:16.342340 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:16.342585 (+ 245us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:16.347788 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:16.347801 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:16.347818 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:16.347872 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:16.353239 (+ 5367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:16.354531 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:16.354545 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:16.354556 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:16.354958 (+ 402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:16.355734 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:16.355740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:16.355744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:16.356676 (+ 932us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:16.357103 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:16.357665 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 22:49:16.358077 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":461,"threads_started":1}
I20260807 22:49:16.363955 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:16.366088 11983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } }
I20260807 22:49:16.366533 11983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:16.366659 11983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2726ea1269474eb2910acd8e0e3c294b, State: Initialized, Role: FOLLOWER
I20260807 22:49:16.367780 11983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } }
I20260807 22:49:16.368198 11983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:16.368402 11983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:16.368729 11983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:16.371703 11983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } }
I20260807 22:49:16.372339 11983 leader_election.cc:302] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2726ea1269474eb2910acd8e0e3c294b; no voters:
I20260807 22:49:16.373143 11983 leader_election.cc:288] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:16.373278 11998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:16.374938 11998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [term 1 LEADER]: Becoming Leader. State: Replica: 2726ea1269474eb2910acd8e0e3c294b, State: Running, Role: LEADER
I20260807 22:49:16.375720 11998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } }
I20260807 22:49:16.376179 11983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:16.379315 12002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2726ea1269474eb2910acd8e0e3c294b. Latest consensus state: current_term: 1 leader_uuid: "2726ea1269474eb2910acd8e0e3c294b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } } }
I20260807 22:49:16.379845 12002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:16.380596 12001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2726ea1269474eb2910acd8e0e3c294b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2726ea1269474eb2910acd8e0e3c294b" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 36221 } } }
I20260807 22:49:16.380991 12001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:16.384617 12008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:16.389048 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:16.368787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58095 (local address 127.20.10.190:36221)
0807 22:49:16.368973 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:16.368977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:16.369583 (+ 606us) server_negotiation.cc:407] Connection header received
0807 22:49:16.369825 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:16.369831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:16.369916 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:16.370052 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:16.371817 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:16.372903 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:16.375038 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:16.375758 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:16.375854 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:16.380079 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:16.380089 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:16.380101 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:16.380151 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:16.385329 (+ 5178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:16.386308 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:16.386315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:16.386319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:16.386479 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:16.387093 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:16.387098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:16.387102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:16.387493 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:16.387852 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:16.388393 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 22:49:16.388663 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:49:16.396167 12008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:16.409124 12008 catalog_manager.cc:1347] Generated new cluster ID: 4cb20f586aee4537a4f22add156b6338
I20260807 22:49:16.409384 12008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:16.424682 12008 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:16.427069 12008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:16.449276 12008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2726ea1269474eb2910acd8e0e3c294b: Generated new TSK 0
I20260807 22:49:16.450559 12008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:16.467058 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 11910
I20260807 22:49:16.469653 11978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:16.571789 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 11910
W20260807 22:49:16.595880 11893 connection.cc:535] client connection to 127.20.10.190:36221 recv error: Network error: failed to read from TLS socket (remote: 127.20.10.190:36221): Connection reset by peer (error 104)
2026-08-07T22:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:16 dist-test-slave-csg5 krb5kdc[12023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:16 dist-test-slave-csg5 krb5kdc[12023](info): set up 2 sockets
Aug 07 22:49:16 dist-test-slave-csg5 krb5kdc[12023](info): commencing operation
krb5kdc: starting...
W20260807 22:49:17.873693 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.003s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:18 dist-test-slave-csg5 krb5kdc[12023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:18.271443 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42677
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:42677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:18.658396 12037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:18.658737 12037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:18.658897 12037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:18.666800 12037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:18.666936 12037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:18.667021 12037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:18.667129 12037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:18.667204 12037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:18.678062 12037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:42677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:18.679617 12037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:18.681069 12037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:18.695603 12043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:18.695537 12046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:18.695493 12044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:18.696133 12037 server_base.cc:1034] running on GCE node
I20260807 22:49:18.697347 12037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:18.699072 12037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:18.700388 12037 hybrid_clock.cc:648] HybridClock initialized: now 1786142958700306 us; error 68 us; skew 500 ppm
I20260807 22:49:18.700870 12037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:18 dist-test-slave-csg5 krb5kdc[12023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142958, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:18.705077 12037 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:18.707104 12037 webserver.cc:460] Webserver started at http://127.20.10.190:41809/ using document root <none> and password file <none>
I20260807 22:49:18.707957 12037 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:18.708129 12037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:18.708493 12037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:18.712095 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "e1aaf2523d934cd59a6d5c014b3c5324"
format_stamp: "Formatted at 2026-08-07 22:49:18 on dist-test-slave-csg5"
server_key: "30c0d8d7340933837e8e6f72d9c781c6"
server_key_iv: "e773a4ddd0d68b9ec9160402415e1842"
server_key_version: "encryptionkey@0"
I20260807 22:49:18.713039 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "e1aaf2523d934cd59a6d5c014b3c5324"
format_stamp: "Formatted at 2026-08-07 22:49:18 on dist-test-slave-csg5"
server_key: "30c0d8d7340933837e8e6f72d9c781c6"
server_key_iv: "e773a4ddd0d68b9ec9160402415e1842"
server_key_version: "encryptionkey@0"
I20260807 22:49:18.719327 12037 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 22:49:18.723577 12054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:18.725188 12037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:49:18.725436 12037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "e1aaf2523d934cd59a6d5c014b3c5324"
format_stamp: "Formatted at 2026-08-07 22:49:18 on dist-test-slave-csg5"
server_key: "30c0d8d7340933837e8e6f72d9c781c6"
server_key_iv: "e773a4ddd0d68b9ec9160402415e1842"
server_key_version: "encryptionkey@0"
I20260807 22:49:18.725698 12037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:18.752332 12037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:18.757171 12037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:18.757503 12037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:18.779309 12037 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:42677
I20260807 22:49:18.779378 12110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:42677 every 8 connection(s)
I20260807 22:49:18.780839 12037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:18.784839 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12037
I20260807 22:49:18.785061 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:18.785357 12111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:18.785698 20522 external_mini_cluster.cc:1428] Setting key 1aeaf2fd1e2319a954a44558f3edabec
I20260807 22:49:18.801964 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:18.787875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50443 (local address 127.20.10.190:42677)
0807 22:49:18.788793 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:18.788809 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:18.788861 (+ 52us) server_negotiation.cc:407] Connection header received
0807 22:49:18.790903 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:18.790934 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:18.791513 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:18.792024 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:18.792591 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:18.794683 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:18.796283 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:18.797371 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:18.797559 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:18.798911 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:18.798922 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:18.798947 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:18.799791 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:18.799903 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:18.799935 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:18.800099 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:18.800397 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 22:49:18.800583 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":401,"threads_started":1}
I20260807 22:49:18.802253 12111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324: Bootstrap starting.
I20260807 22:49:18.805822 12111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:18.806746 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:18.807152 12111 log.cc:824] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:18.810961 12111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324: No bootstrap required, opened a new log
I20260807 22:49:18.819875 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:18.812251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41083 (local address 127.20.10.190:42677)
0807 22:49:18.812427 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:18.812434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:18.812824 (+ 390us) server_negotiation.cc:407] Connection header received
0807 22:49:18.812998 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:18.813005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:18.813096 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:18.813212 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:18.813883 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:18.815114 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:18.816567 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:18.817314 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:18.817454 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:18.818806 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:18.818812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:18.818818 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:18.818842 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:18.818892 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:18.818913 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:18.819057 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:18.819421 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 22:49:18.819481 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:49:18.821812 12111 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } }
I20260807 22:49:18.822209 12111 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:18.822288 12111 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1aaf2523d934cd59a6d5c014b3c5324, State: Initialized, Role: FOLLOWER
I20260807 22:49:18.822883 12111 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } }
I20260807 22:49:18.823117 12111 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:18.823275 12111 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:18.823467 12111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:18.826200 12111 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } }
I20260807 22:49:18.826848 12111 leader_election.cc:302] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1aaf2523d934cd59a6d5c014b3c5324; no voters:
I20260807 22:49:18.827551 12111 leader_election.cc:288] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:18.827775 12123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:18.828758 12123 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [term 1 LEADER]: Becoming Leader. State: Replica: e1aaf2523d934cd59a6d5c014b3c5324, State: Running, Role: LEADER
I20260807 22:49:18.829628 12123 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } }
I20260807 22:49:18.830313 12111 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:18.832842 12124 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } } }
I20260807 22:49:18.833312 12125 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1aaf2523d934cd59a6d5c014b3c5324. Latest consensus state: current_term: 1 leader_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aaf2523d934cd59a6d5c014b3c5324" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42677 } } }
I20260807 22:49:18.833506 12124 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:18.833591 12125 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:18.834794 12133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:18.841464 12133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:18.850790 12133 catalog_manager.cc:1347] Generated new cluster ID: 6027f9e459794e62b75d91c634b53366
I20260807 22:49:18.850956 12133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:18.864691 12133 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:18.866712 12133 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:18.878188 12133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1aaf2523d934cd59a6d5c014b3c5324: Generated new TSK 0
I20260807 22:49:18.879458 12133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:18.893268 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12037
W20260807 22:49:18.894637 12106 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50443
W20260807 22:49:18.895587 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12037: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:18.895777 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12037
W20260807 22:49:18.919828 12021 connection.cc:535] client connection to 127.20.10.190:42677 recv error: Network error: failed to read from TLS socket (remote: 127.20.10.190:42677): Connection reset by peer (error 104)
2026-08-07T22:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:19 dist-test-slave-csg5 krb5kdc[12146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:19 dist-test-slave-csg5 krb5kdc[12146](info): set up 2 sockets
Aug 07 22:49:19 dist-test-slave-csg5 krb5kdc[12146](info): commencing operation
krb5kdc: starting...
W20260807 22:49:20.180387 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.005s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:20 dist-test-slave-csg5 krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:20.584373 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42329
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:42329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:20.968583 12160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:20.969015 12160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:20.969199 12160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:20.976977 12160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:20.977174 12160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:20.977304 12160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:20.977413 12160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:20.977504 12160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:20.988276 12160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:42329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:42329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:20.989817 12160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:20.991307 12160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:21.004617 12166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:21.004793 12165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:21.004808 12168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:21.005545 12160 server_base.cc:1034] running on GCE node
I20260807 22:49:21.006489 12160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:21.008118 12160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:21.009397 12160 hybrid_clock.cc:648] HybridClock initialized: now 1786142961009338 us; error 44 us; skew 500 ppm
I20260807 22:49:21.009843 12160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:21 dist-test-slave-csg5 krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:21.013859 12160 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:21.015830 12160 webserver.cc:460] Webserver started at http://127.20.10.190:37137/ using document root <none> and password file <none>
I20260807 22:49:21.016618 12160 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:21.016763 12160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:21.017114 12160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:21.020533 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "819c15beee3446e8ae12fa170bdb05f9"
format_stamp: "Formatted at 2026-08-07 22:49:21 on dist-test-slave-csg5"
server_key: "ff3eb68429b89ceb8b36b685e7d0eafd"
server_key_iv: "2533f0c7c34100371eef25d1fa54b142"
server_key_version: "encryptionkey@0"
I20260807 22:49:21.021399 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "819c15beee3446e8ae12fa170bdb05f9"
format_stamp: "Formatted at 2026-08-07 22:49:21 on dist-test-slave-csg5"
server_key: "ff3eb68429b89ceb8b36b685e7d0eafd"
server_key_iv: "2533f0c7c34100371eef25d1fa54b142"
server_key_version: "encryptionkey@0"
I20260807 22:49:21.027612 12160 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 22:49:21.031980 12176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:21.033831 12160 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:49:21.034040 12160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "819c15beee3446e8ae12fa170bdb05f9"
format_stamp: "Formatted at 2026-08-07 22:49:21 on dist-test-slave-csg5"
server_key: "ff3eb68429b89ceb8b36b685e7d0eafd"
server_key_iv: "2533f0c7c34100371eef25d1fa54b142"
server_key_version: "encryptionkey@0"
I20260807 22:49:21.034284 12160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:21.041093 12160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:21.046031 12160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:21.046383 12160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:21.068462 12160 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:42329
I20260807 22:49:21.068509 12231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:42329 every 8 connection(s)
I20260807 22:49:21.069968 12160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:21.074090 12232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:21.074741 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12160
I20260807 22:49:21.074945 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:21.075524 20522 external_mini_cluster.cc:1428] Setting key d5149cae0392b6c1a11c9cafcdfac0d7
I20260807 22:49:21.086073 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:21.077703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38469 (local address 127.20.10.190:42329)
0807 22:49:21.078901 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:21.078921 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:21.079331 (+ 410us) server_negotiation.cc:407] Connection header received
0807 22:49:21.081862 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:21.081894 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:21.082377 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:21.082955 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:21.084199 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:21.084206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:21.084220 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:21.084238 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:21.084302 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:21.084322 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:21.084615 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:21.084768 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:49:21.084984 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":541,"threads_started":1}
I20260807 22:49:21.089952 12232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9: Bootstrap starting.
I20260807 22:49:21.090749 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:21.093354 12232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:21.094594 12232 log.cc:824] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:21.098485 12232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9: No bootstrap required, opened a new log
I20260807 22:49:21.098812 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:21.095626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44963 (local address 127.20.10.190:42329)
0807 22:49:21.095828 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:21.095835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:21.096635 (+ 800us) server_negotiation.cc:407] Connection header received
0807 22:49:21.096783 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:21.096788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:21.096859 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:21.096985 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:21.098009 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:21.098016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:21.098021 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:21.098041 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:21.098091 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:21.098112 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:21.098243 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:21.098421 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:49:21.098509 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:49:21.106936 12232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } }
I20260807 22:49:21.107486 12232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:21.107631 12232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819c15beee3446e8ae12fa170bdb05f9, State: Initialized, Role: FOLLOWER
I20260807 22:49:21.108358 12232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } }
I20260807 22:49:21.108618 12232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:21.108803 12232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:21.108979 12232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:21.111792 12232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } }
I20260807 22:49:21.112387 12232 leader_election.cc:302] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 819c15beee3446e8ae12fa170bdb05f9; no voters:
I20260807 22:49:21.113199 12232 leader_election.cc:288] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:21.113411 12244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:21.114221 12244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [term 1 LEADER]: Becoming Leader. State: Replica: 819c15beee3446e8ae12fa170bdb05f9, State: Running, Role: LEADER
I20260807 22:49:21.114889 12244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } }
I20260807 22:49:21.115729 12232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:21.118525 12245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "819c15beee3446e8ae12fa170bdb05f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } } }
I20260807 22:49:21.118499 12246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 819c15beee3446e8ae12fa170bdb05f9. Latest consensus state: current_term: 1 leader_uuid: "819c15beee3446e8ae12fa170bdb05f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819c15beee3446e8ae12fa170bdb05f9" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 42329 } } }
I20260807 22:49:21.119144 12246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:21.119148 12245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:21.121527 12252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:21.128141 12252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:21.138909 12252 catalog_manager.cc:1347] Generated new cluster ID: 7f05abe50e5349319be06d1ebd50afd3
I20260807 22:49:21.139214 12252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:21.168463 12252 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:21.168994 12252 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:21.177618 12252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 819c15beee3446e8ae12fa170bdb05f9: Generated new TSK 0
I20260807 22:49:21.178855 12252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:21.192790 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12160
W20260807 22:49:21.194443 12226 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38469
W20260807 22:49:21.195367 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12160: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:21.195529 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12160
2026-08-07T22:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:21 dist-test-slave-csg5 krb5kdc[12268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:21 dist-test-slave-csg5 krb5kdc[12268](info): set up 2 sockets
Aug 07 22:49:21 dist-test-slave-csg5 krb5kdc[12268](info): commencing operation
krb5kdc: starting...
W20260807 22:49:22.453509 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:22 dist-test-slave-csg5 krb5kdc[12268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:22.852175 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33049
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:23.238131 12282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:23.238487 12282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:23.238600 12282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:23.246464 12282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:23.246702 12282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:23.246790 12282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:23.246860 12282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:23.246907 12282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:23.257903 12282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:23.259405 12282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:23.260836 12282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:23.274325 12287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:23.274430 12289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:23.274488 12292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:23.275933 12282 server_base.cc:1034] running on GCE node
I20260807 22:49:23.276860 12282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:23.278498 12282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:23.279747 12282 hybrid_clock.cc:648] HybridClock initialized: now 1786142963279706 us; error 50 us; skew 500 ppm
I20260807 22:49:23.280248 12282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:23 dist-test-slave-csg5 krb5kdc[12268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:23.291445 12282 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:23.293427 12282 webserver.cc:460] Webserver started at http://127.20.10.190:39381/ using document root <none> and password file <none>
I20260807 22:49:23.294267 12282 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:23.294453 12282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:23.294869 12282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:23.299795 12282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "599ca664c594491d897c195261e19d06"
format_stamp: "Formatted at 2026-08-07 22:49:23 on dist-test-slave-csg5"
server_key: "bafe55aa1c2096f58ebc853ff3eb0816"
server_key_iv: "0fcf63d06c14448a1becc4c29f7a10d8"
server_key_version: "encryptionkey@0"
I20260807 22:49:23.300779 12282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "599ca664c594491d897c195261e19d06"
format_stamp: "Formatted at 2026-08-07 22:49:23 on dist-test-slave-csg5"
server_key: "bafe55aa1c2096f58ebc853ff3eb0816"
server_key_iv: "0fcf63d06c14448a1becc4c29f7a10d8"
server_key_version: "encryptionkey@0"
I20260807 22:49:23.307394 12282 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 22:49:23.311878 12300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:23.313792 12282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:49:23.314057 12282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "599ca664c594491d897c195261e19d06"
format_stamp: "Formatted at 2026-08-07 22:49:23 on dist-test-slave-csg5"
server_key: "bafe55aa1c2096f58ebc853ff3eb0816"
server_key_iv: "0fcf63d06c14448a1becc4c29f7a10d8"
server_key_version: "encryptionkey@0"
I20260807 22:49:23.314325 12282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:23.334594 12282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:23.340087 12282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:23.340373 12282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:23.363283 12282 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33049
I20260807 22:49:23.363327 12356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33049 every 8 connection(s)
I20260807 22:49:23.364812 12282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:23.369266 12357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:23.372939 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12282
I20260807 22:49:23.373143 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:23.373624 20522 external_mini_cluster.cc:1428] Setting key 90d47f80360abcdfa496af15d9c1223c
I20260807 22:49:23.383792 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06: Bootstrap starting.
I20260807 22:49:23.388430 12357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:23.390066 12357 log.cc:824] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:49:23 dist-test-slave-csg5 krb5kdc[12268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:23.394330 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06: No bootstrap required, opened a new log
I20260807 22:49:23.406111 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:23.375949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40245 (local address 127.20.10.190:33049)
0807 22:49:23.376839 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:23.376851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:23.376895 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:49:23.378745 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:23.378772 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:23.379257 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:23.379616 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:23.381688 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:23.383314 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:23.385496 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:23.386927 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:23.387154 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:23.393217 (+ 6063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:23.393231 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:23.393253 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:23.393315 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:23.398228 (+ 4913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:23.399474 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:23.399486 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:23.399497 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:23.399947 (+ 450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:23.400887 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:23.400894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:23.400899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:23.402011 (+ 1112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:23.402857 (+ 846us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:23.403426 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 22:49:23.404045 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":429,"threads_started":1}
I20260807 22:49:23.409200 12357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } }
I20260807 22:49:23.409888 12357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:23.410173 12357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599ca664c594491d897c195261e19d06, State: Initialized, Role: FOLLOWER
I20260807 22:49:23.412098 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:23.412286 12357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } }
I20260807 22:49:23.413211 12357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:23.413511 12357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:23.413753 12357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:23.417788 12357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } }
I20260807 22:49:23.418720 12357 leader_election.cc:302] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 599ca664c594491d897c195261e19d06; no voters:
I20260807 22:49:23.419541 12357 leader_election.cc:288] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:23.419756 12367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:23.420559 12367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [term 1 LEADER]: Becoming Leader. State: Replica: 599ca664c594491d897c195261e19d06, State: Running, Role: LEADER
I20260807 22:49:23.421496 12367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } }
I20260807 22:49:23.424861 12357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:23.434012 12368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 599ca664c594491d897c195261e19d06. Latest consensus state: current_term: 1 leader_uuid: "599ca664c594491d897c195261e19d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } } }
I20260807 22:49:23.434913 12368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:23.435904 12369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "599ca664c594491d897c195261e19d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599ca664c594491d897c195261e19d06" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33049 } } }
I20260807 22:49:23.436343 12369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:23.444796 12377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:23.455433 12377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:23.458750 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:23.421171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45537 (local address 127.20.10.190:33049)
0807 22:49:23.421413 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:23.421423 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:23.431933 (+ 10510us) server_negotiation.cc:407] Connection header received
0807 22:49:23.433142 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:23.433157 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:23.433406 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:23.433569 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:23.436642 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:23.438142 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:23.440957 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:23.441963 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:23.442111 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:23.446423 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:23.446434 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:23.446439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:23.446481 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:23.452366 (+ 5885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:23.453339 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:23.453347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:23.453351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:23.453686 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:23.454346 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:23.454352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:23.454356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:23.455161 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:23.457388 (+ 2227us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:23.457722 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 22:49:23.458106 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:49:23.468760 12377 catalog_manager.cc:1347] Generated new cluster ID: 9e912f4a36504065bf805b117be81ed9
I20260807 22:49:23.468914 12377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:23.493682 12377 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:23.495471 12377 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:23.507717 12377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 599ca664c594491d897c195261e19d06: Generated new TSK 0
I20260807 22:49:23.508922 12377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:23.522675 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12282
I20260807 22:49:23.524050 12352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:23.623246 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12282
2026-08-07T22:49:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:23 dist-test-slave-csg5 krb5kdc[12394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:23 dist-test-slave-csg5 krb5kdc[12394](info): set up 2 sockets
Aug 07 22:49:23 dist-test-slave-csg5 krb5kdc[12394](info): commencing operation
krb5kdc: starting...
W20260807 22:49:24.854162 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:25 dist-test-slave-csg5 krb5kdc[12394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:25.250408 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46015
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:46015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:25.641497 12408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:25.641830 12408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:25.641993 12408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:25.649719 12408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:25.649852 12408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:25.649968 12408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:25.650063 12408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:25.650125 12408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:25.662478 12408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:46015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:46015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:25.664050 12408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:25.665545 12408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:25.679579 12416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:25.679649 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:25.679653 12415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:25.681408 12408 server_base.cc:1034] running on GCE node
I20260807 22:49:25.682389 12408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:25.684118 12408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:25.685465 12408 hybrid_clock.cc:648] HybridClock initialized: now 1786142965685433 us; error 55 us; skew 500 ppm
I20260807 22:49:25.685921 12408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:25 dist-test-slave-csg5 krb5kdc[12394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:25.690254 12408 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:25.692322 12408 webserver.cc:460] Webserver started at http://127.20.10.190:42223/ using document root <none> and password file <none>
I20260807 22:49:25.693151 12408 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:25.693286 12408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:25.693624 12408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:25.697470 12408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "98a05a1333af43c2bb534db35d30d9df"
format_stamp: "Formatted at 2026-08-07 22:49:25 on dist-test-slave-csg5"
server_key: "638af8374cce735edc66e79820e40e2b"
server_key_iv: "e30bcfb2824eaa2fc81c859e94e54182"
server_key_version: "encryptionkey@0"
I20260807 22:49:25.698632 12408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "98a05a1333af43c2bb534db35d30d9df"
format_stamp: "Formatted at 2026-08-07 22:49:25 on dist-test-slave-csg5"
server_key: "638af8374cce735edc66e79820e40e2b"
server_key_iv: "e30bcfb2824eaa2fc81c859e94e54182"
server_key_version: "encryptionkey@0"
I20260807 22:49:25.706210 12408 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260807 22:49:25.711061 12426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:25.712904 12408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:49:25.713191 12408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "98a05a1333af43c2bb534db35d30d9df"
format_stamp: "Formatted at 2026-08-07 22:49:25 on dist-test-slave-csg5"
server_key: "638af8374cce735edc66e79820e40e2b"
server_key_iv: "e30bcfb2824eaa2fc81c859e94e54182"
server_key_version: "encryptionkey@0"
I20260807 22:49:25.713471 12408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:25.746508 12408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:25.751863 12408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:25.752334 12408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:25.775848 12408 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:46015
I20260807 22:49:25.775849 12483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:46015 every 8 connection(s)
I20260807 22:49:25.777511 12408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:25.781066 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12408
I20260807 22:49:25.781270 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:25.781842 20522 external_mini_cluster.cc:1428] Setting key 49a0d21d66e45974f64ccdb20ace2401
I20260807 22:49:25.782089 12484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:25.798589 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df: Bootstrap starting.
I20260807 22:49:25.798802 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:25.783898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38045 (local address 127.20.10.190:46015)
0807 22:49:25.784835 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:25.784849 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:25.784900 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:49:25.787301 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:25.787328 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:25.787810 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:25.788193 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:25.788946 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:25.790470 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:25.792522 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:25.793421 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:25.793592 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:25.795735 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:25.795744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:25.795763 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:25.796632 (+ 869us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:25.796744 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:25.796781 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:25.796894 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:25.797245 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 22:49:25.797620 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":457,"threads_started":1}
I20260807 22:49:25.802222 12484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:25.803364 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:25.803720 12484 log.cc:824] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:25.807776 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df: No bootstrap required, opened a new log
I20260807 22:49:25.817273 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:25.809559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46995 (local address 127.20.10.190:46015)
0807 22:49:25.809774 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:25.809783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:25.810373 (+ 590us) server_negotiation.cc:407] Connection header received
0807 22:49:25.810541 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:25.810548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:25.810642 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:25.810812 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:25.811355 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:25.812485 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:25.814154 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:25.814934 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:25.815120 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:25.816370 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:25.816376 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:25.816381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:25.816413 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:25.816470 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:25.816491 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:25.816601 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:25.816793 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 22:49:25.816904 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:49:25.818053 12484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } }
I20260807 22:49:25.818638 12484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:25.818785 12484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a05a1333af43c2bb534db35d30d9df, State: Initialized, Role: FOLLOWER
I20260807 22:49:25.819499 12484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } }
I20260807 22:49:25.819767 12484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:25.819954 12484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:25.820118 12484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:25.822847 12484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } }
I20260807 22:49:25.823429 12484 leader_election.cc:302] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98a05a1333af43c2bb534db35d30d9df; no voters:
I20260807 22:49:25.824122 12484 leader_election.cc:288] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:25.824350 12499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:25.825361 12499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [term 1 LEADER]: Becoming Leader. State: Replica: 98a05a1333af43c2bb534db35d30d9df, State: Running, Role: LEADER
I20260807 22:49:25.826018 12499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } }
I20260807 22:49:25.826712 12484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:25.829335 12500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98a05a1333af43c2bb534db35d30d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } } }
I20260807 22:49:25.829694 12501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98a05a1333af43c2bb534db35d30d9df. Latest consensus state: current_term: 1 leader_uuid: "98a05a1333af43c2bb534db35d30d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a05a1333af43c2bb534db35d30d9df" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 46015 } } }
I20260807 22:49:25.830003 12500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:25.829979 12501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:25.832161 12505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:25.838032 12505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:25.849694 12505 catalog_manager.cc:1347] Generated new cluster ID: 7ea1fa2eed004679a60d0325baf02c08
I20260807 22:49:25.849859 12505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:25.858311 12505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:25.859779 12505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:25.872663 12505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98a05a1333af43c2bb534db35d30d9df: Generated new TSK 0
I20260807 22:49:25.873618 12505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:25.880188 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12408
W20260807 22:49:25.881642 12478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38045
W20260807 22:49:25.882519 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12408: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:25.882740 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12408
2026-08-07T22:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2258 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:26 dist-test-slave-csg5 krb5kdc[12521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:26 dist-test-slave-csg5 krb5kdc[12521](info): set up 2 sockets
Aug 07 22:49:26 dist-test-slave-csg5 krb5kdc[12521](info): commencing operation
krb5kdc: starting...
W20260807 22:49:27.164902 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:27 dist-test-slave-csg5 krb5kdc[12521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:27.568221 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33909
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:27.953836 12537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:27.954173 12537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:27.954274 12537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:27.962217 12537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:27.962371 12537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:27.962420 12537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:27.962481 12537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:27.962527 12537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:27.973107 12537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:27.974515 12537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:27.976054 12537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:27.989760 12542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:27.990015 12543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:27.989832 12546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:27.991147 12537 server_base.cc:1034] running on GCE node
I20260807 22:49:27.992127 12537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:27.993865 12537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:27.995119 12537 hybrid_clock.cc:648] HybridClock initialized: now 1786142967994988 us; error 126 us; skew 500 ppm
I20260807 22:49:27.995647 12537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:27 dist-test-slave-csg5 krb5kdc[12521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:27.999459 12537 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:28.001583 12537 webserver.cc:460] Webserver started at http://127.20.10.190:46521/ using document root <none> and password file <none>
I20260807 22:49:28.002326 12537 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:28.002441 12537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:28.002822 12537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:28.006193 12537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "6a08e2165063448487b66054c84c511d"
format_stamp: "Formatted at 2026-08-07 22:49:28 on dist-test-slave-csg5"
server_key: "6d185b14337cc0178a59a93285183f01"
server_key_iv: "6345169a6933991a471c25278e66ac68"
server_key_version: "encryptionkey@0"
I20260807 22:49:28.007125 12537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "6a08e2165063448487b66054c84c511d"
format_stamp: "Formatted at 2026-08-07 22:49:28 on dist-test-slave-csg5"
server_key: "6d185b14337cc0178a59a93285183f01"
server_key_iv: "6345169a6933991a471c25278e66ac68"
server_key_version: "encryptionkey@0"
I20260807 22:49:28.013249 12537 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 22:49:28.017802 12553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:28.019516 12537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 22:49:28.019830 12537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "6a08e2165063448487b66054c84c511d"
format_stamp: "Formatted at 2026-08-07 22:49:28 on dist-test-slave-csg5"
server_key: "6d185b14337cc0178a59a93285183f01"
server_key_iv: "6345169a6933991a471c25278e66ac68"
server_key_version: "encryptionkey@0"
I20260807 22:49:28.020063 12537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:28.027743 12537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:28.033427 12537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:28.033736 12537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:28.056264 12537 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:33909
I20260807 22:49:28.056275 12609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:33909 every 8 connection(s)
I20260807 22:49:28.057839 12537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:28.059065 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12537
I20260807 22:49:28.059260 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:28.059706 20522 external_mini_cluster.cc:1428] Setting key 4732713e1956ea3da0738318af32152b
I20260807 22:49:28.062481 12611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:28.068192 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:28.061638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54159 (local address 127.20.10.190:33909)
0807 22:49:28.062547 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:28.062560 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:28.062633 (+ 73us) server_negotiation.cc:407] Connection header received
0807 22:49:28.064362 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:28.064384 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:28.064722 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:28.065168 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:28.066376 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:28.066386 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:28.066402 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:28.066418 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:28.066493 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:28.066520 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:28.066810 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:28.066956 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 22:49:28.067200 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":352,"threads_started":1}
I20260807 22:49:28.072513 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:28.074645 12611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d: Bootstrap starting.
I20260807 22:49:28.079290 12611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:28.080736 12611 log.cc:824] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:28.081626 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:28.077991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35249 (local address 127.20.10.190:33909)
0807 22:49:28.078276 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:28.078287 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:28.078961 (+ 674us) server_negotiation.cc:407] Connection header received
0807 22:49:28.079159 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:28.079167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:28.079245 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:28.079407 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:28.080531 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:28.080537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:28.080542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:49:28.080564 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:28.080618 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:49:28.080641 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:28.080806 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:28.081098 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 22:49:28.081181 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:49:28.084849 12611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d: No bootstrap required, opened a new log
I20260807 22:49:28.093904 12611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } }
I20260807 22:49:28.094368 12611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:49:28.094478 12611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a08e2165063448487b66054c84c511d, State: Initialized, Role: FOLLOWER
I20260807 22:49:28.095285 12611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } }
I20260807 22:49:28.095584 12611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:49:28.095806 12611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:49:28.096033 12611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:28.098408 12611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } }
I20260807 22:49:28.098989 12611 leader_election.cc:302] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a08e2165063448487b66054c84c511d; no voters:
I20260807 22:49:28.099689 12611 leader_election.cc:288] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:49:28.100005 12624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:28.100791 12624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [term 1 LEADER]: Becoming Leader. State: Replica: 6a08e2165063448487b66054c84c511d, State: Running, Role: LEADER
I20260807 22:49:28.101400 12624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } }
I20260807 22:49:28.102257 12611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:28.105019 12626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a08e2165063448487b66054c84c511d. Latest consensus state: current_term: 1 leader_uuid: "6a08e2165063448487b66054c84c511d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } } }
I20260807 22:49:28.105499 12626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:28.106910 12625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a08e2165063448487b66054c84c511d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a08e2165063448487b66054c84c511d" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 33909 } } }
I20260807 22:49:28.107293 12625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:28.108547 12632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:28.114362 12632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:28.124987 12632 catalog_manager.cc:1347] Generated new cluster ID: 3bf0e9b6c1894ecea39b1be8cbabec39
I20260807 22:49:28.125167 12632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:28.148125 12632 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:28.148700 12632 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:28.155616 12632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a08e2165063448487b66054c84c511d: Generated new TSK 0
I20260807 22:49:28.156796 12632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:49:28.176465 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12537
W20260807 22:49:28.177620 12605 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54159
W20260807 22:49:28.178419 20522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12537: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 22:49:28.178567 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12537
2026-08-07T22:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2294 ms)
[----------] 18 tests from AuthTokenIssuingTest (42048 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:28 dist-test-slave-csg5 krb5kdc[12647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:28 dist-test-slave-csg5 krb5kdc[12647](info): set up 2 sockets
Aug 07 22:49:28 dist-test-slave-csg5 krb5kdc[12647](info): commencing operation
krb5kdc: starting...
W20260807 22:49:29.470152 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.002s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:29 dist-test-slave-csg5 krb5kdc[12647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:29.880867 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44107
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:30.264940 12660 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:30.265442 12660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:30.265574 12660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:30.265733 12660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:30.273708 12660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:30.273871 12660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:30.273979 12660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:30.274080 12660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:30.274180 12660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:30.284812 12660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:44107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:30.286383 12660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:30.287878 12660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:30.302428 12668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:30.302503 12666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:30.302700 12665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:30.303777 12660 server_base.cc:1034] running on GCE node
I20260807 22:49:30.304832 12660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:30.306578 12660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:30.307921 12660 hybrid_clock.cc:648] HybridClock initialized: now 1786142970307852 us; error 47 us; skew 500 ppm
I20260807 22:49:30.308353 12660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:30 dist-test-slave-csg5 krb5kdc[12647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:30.318080 12660 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:30.320080 12660 webserver.cc:460] Webserver started at http://127.20.10.190:36701/ using document root <none> and password file <none>
I20260807 22:49:30.320914 12660 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:30.321072 12660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:30.321439 12660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:30.325361 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "3d10aa6566ca481bbda8f273a6867b39"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "021242be9e9d1c28c65f17b98758684e"
server_key_iv: "1d679787de9bc0b068dbc0d664b07625"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.326303 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "3d10aa6566ca481bbda8f273a6867b39"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "021242be9e9d1c28c65f17b98758684e"
server_key_iv: "1d679787de9bc0b068dbc0d664b07625"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.332600 12660 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 22:49:30.336849 12675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:30.338639 12660 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 22:49:30.338937 12660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "3d10aa6566ca481bbda8f273a6867b39"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "021242be9e9d1c28c65f17b98758684e"
server_key_iv: "1d679787de9bc0b068dbc0d664b07625"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.339211 12660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:30.369711 12660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:30.375146 12660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:30.375473 12660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:30.397562 12660 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:44107
I20260807 22:49:30.397586 12732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:44107 every 8 connection(s)
I20260807 22:49:30.399290 12660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:30.403801 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:30.404037 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12660
I20260807 22:49:30.404297 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:30.405021 20522 external_mini_cluster.cc:1428] Setting key 28386894b4b73602ec753d93ad724264
I20260807 22:49:30.412001 12733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:30 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:30.442044 12735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.416004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.20.10.190:44107)
0807 22:49:30.417027 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:30.417047 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:30.417732 (+ 685us) server_negotiation.cc:407] Connection header received
0807 22:49:30.420929 (+ 3197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:30.420936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:30.421230 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:30.421411 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:30.423533 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:30.424908 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:30.427654 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:30.428313 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:30.428476 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:30.433799 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:30.433809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:30.433830 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:30.433866 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:30.437107 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:30.437902 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:30.437909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:30.437911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:30.438066 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:30.438492 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:30.438508 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:30.438510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:30.439246 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:30.439468 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:30.440068 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 22:49:30.440421 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":451,"threads_started":1}
I20260807 22:49:30.442023 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.416004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44107 (local address 127.0.0.1:60542)
0807 22:49:30.417508 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.417566 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:30.420424 (+ 2858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:30.421539 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:30.421554 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:30.422639 (+ 1085us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:30.423243 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:30.423286 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:30.425141 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:30.425150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:30.427461 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:30.427483 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:30.428650 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:30.428666 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:30.428962 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:30.429796 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:30.429836 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:30.433444 (+ 3608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:30.437275 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:30.437290 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:30.437304 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:30.437745 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:30.438198 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:30.438203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:30.438207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:30.438373 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:30.439381 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:30.439389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:30.439633 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 22:49:30.439928 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 22:49:30.440558 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":122,"spinlock_wait_cycles":87424,"thread_start_us":933,"threads_started":1}
WARNING: no policy specified for kudu/127.20.10.189@KRBTEST.COM; defaulting to no policy
I20260807 22:49:30.445530 12733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:30.447044 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.446517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42396)
0807 22:49:30.446744 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.446900 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.189@KRBTEST.COM" created.
W20260807 22:49:30.447397 12677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
I20260807 22:49:30.449482 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.448981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42404)
0807 22:49:30.449230 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.449321 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:49:30.449870 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:46733: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
I20260807 22:49:30.491326 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:46733
--webserver_interface=127.20.10.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:49:30.499387 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } attempt: 1
I20260807 22:49:30.500809 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.500234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42406)
0807 22:49:30.500506 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.500656 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 22:49:30.501948 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.501541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42418)
0807 22:49:30.501727 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.501803 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 22:49:30.502182 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:46733: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
I20260807 22:49:30.570505 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } attempt: 2
I20260807 22:49:30.572078 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.571451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42420)
0807 22:49:30.571764 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.571896 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:49:30.573369 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.572935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42428)
0807 22:49:30.573135 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.573218 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 22:49:30.573665 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:46733: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
I20260807 22:49:30.665024 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } attempt: 3
I20260807 22:49:30.666349 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.665862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42440)
0807 22:49:30.666086 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.666199 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:49:30.667665 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.667141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42450)
0807 22:49:30.667419 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.667510 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260807 22:49:30.667877 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:46733: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
I20260807 22:49:30.811167 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } attempt: 4
I20260807 22:49:30.812479 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.811890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42464)
0807 22:49:30.812236 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.812324 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260807 22:49:30.813699 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:30.813294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42470)
0807 22:49:30.813453 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:30.813531 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:49:30.813882 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:46733: Network error: Client connection negotiation failed: client connection to 127.20.10.189:46733: connect: Connection refused (error 111)
W20260807 22:49:30.880138 12740 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:30.880566 12740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:30.880697 12740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:30.880839 12740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:30.888700 12740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:30.888859 12740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:30.888983 12740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:30.889078 12740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:30.889204 12740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:30.900573 12740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:46733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.10.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:30.902076 12740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:30.903594 12740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:30.917441 12746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:30.917431 12748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:30.917701 12745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:30.919683 12740 server_base.cc:1034] running on GCE node
I20260807 22:49:30.920749 12740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:30.922628 12740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:30.924023 12740 hybrid_clock.cc:648] HybridClock initialized: now 1786142970923919 us; error 83 us; skew 500 ppm
I20260807 22:49:30.924737 12740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:30 dist-test-slave-csg5 krb5kdc[12647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:30.928679 12740 init.cc:377] Logged in from keytab as kudu/127.20.10.189@KRBTEST.COM (short username kudu)
I20260807 22:49:30.930608 12740 webserver.cc:460] Webserver started at http://127.20.10.189:34871/ using document root <none> and password file <none>
I20260807 22:49:30.931762 12740 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:30.931921 12740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:30.932308 12740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:30.935931 12740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data/instance:
uuid: "7c50c5e7f0e04c978370ad61d2e092a2"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "6ac83fbc4b05c35f97f3eee4c947a732"
server_key_iv: "0790228e892825c05bef40c2fe4c3cd3"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.936894 12740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal/instance:
uuid: "7c50c5e7f0e04c978370ad61d2e092a2"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "6ac83fbc4b05c35f97f3eee4c947a732"
server_key_iv: "0790228e892825c05bef40c2fe4c3cd3"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.943444 12740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 22:49:30.948014 12757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:30.949870 12740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:49:30.950129 12740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "7c50c5e7f0e04c978370ad61d2e092a2"
format_stamp: "Formatted at 2026-08-07 22:49:30 on dist-test-slave-csg5"
server_key: "6ac83fbc4b05c35f97f3eee4c947a732"
server_key_iv: "0790228e892825c05bef40c2fe4c3cd3"
server_key_version: "encryptionkey@0"
I20260807 22:49:30.950415 12740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:30.966343 12740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:30.971786 12740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:30.972188 12740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:30.995806 12740 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.189:46733
I20260807 22:49:30.995838 12814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.189:46733 every 8 connection(s)
I20260807 22:49:30.997292 12740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
I20260807 22:49:31.001660 12815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:31.003368 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12740
I20260807 22:49:31.003602 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-1/wal/instance
I20260807 22:49:31.004226 20522 external_mini_cluster.cc:1428] Setting key 40e21596612fe975bdd9c4cee36d8d18
I20260807 22:49:31.013310 12815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:31.040774 12817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.017417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.20.10.190:44107)
0807 22:49:31.018030 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.018040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.019209 (+ 1169us) server_negotiation.cc:407] Connection header received
0807 22:49:31.021597 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.021607 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.021776 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.021953 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.024001 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.024821 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.026681 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.027511 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.027704 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.032829 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.032836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.032840 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.032895 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.036857 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.037866 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.037871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.037874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.038012 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.038543 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.038547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.038549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.039028 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.039209 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.040084 (+ 875us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.040359 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":285,"threads_started":1}
I20260807 22:49:31.041874 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.018065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44107 (local address 127.0.0.1:60554)
0807 22:49:31.019000 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.019059 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.021227 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.022266 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.022277 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.023005 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.023710 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.023732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.025037 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.025042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.026467 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.026482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.027847 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.027857 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.028131 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.029115 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.029142 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.032478 (+ 3336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.037103 (+ 4625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.037114 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.037132 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.037693 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.038165 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.038169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.038171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.038396 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.039197 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.039205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.039651 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.039949 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.040543 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":407,"threads_started":1}
WARNING: no policy specified for kudu/127.20.10.188@KRBTEST.COM; defaulting to no policy
I20260807 22:49:31.044548 12815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.188@KRBTEST.COM" created.
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:31.062453 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.045570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42484)
0807 22:49:31.045803 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.045821 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.045924 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.047590 (+ 1666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.047595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.048127 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.048279 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.048289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.049936 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.049945 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.050988 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.051002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.051976 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.051982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.052244 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.052976 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.052996 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.055603 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.059559 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.059566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.059572 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.060000 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.060428 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.060434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.060438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.060611 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.061312 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.061317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.061485 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.061645 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.061980 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":5}
I20260807 22:49:31.062582 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.046025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.20.10.189:46733)
0807 22:49:31.046908 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.046921 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.046960 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:49:31.047038 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.047049 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.047391 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.047606 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.048507 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.049771 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.051160 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.051805 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.051963 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.055872 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.055887 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.055902 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.055936 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.059397 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.060165 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.060174 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.060176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.060307 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.060729 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.060736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.060739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.061172 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.061406 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.061777 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.062141 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":647,"thread_start_us":430,"threads_started":1}
I20260807 22:49:31.066023 12815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:31.067515 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.066809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59834)
0807 22:49:31.067055 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.067246 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 22:49:31.067854 12759 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.070170 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.069673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59850)
0807 22:49:31.069885 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.069997 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 22:49:31.071115 12815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
I20260807 22:49:31.095243 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:36625
--webserver_interface=127.20.10.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:49:31.113114 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } attempt: 5
I20260807 22:49:31.120498 12815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 1
I20260807 22:49:31.121740 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.121248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59854)
0807 22:49:31.121460 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.121573 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:31.123208 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.122576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59858)
0807 22:49:31.122888 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.123013 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260807 22:49:31.123538 12815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.131816 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.114215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.20.10.189:46733)
0807 22:49:31.114428 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.114433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.114454 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:49:31.114517 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.114521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.114591 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.114747 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.115930 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.116691 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.118642 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.119417 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.119541 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.124255 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.124265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.124272 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.124324 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.128493 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.129573 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.129579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.129583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.129741 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.130181 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.130186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.130191 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.130506 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.130702 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.131146 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.131412 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:49:31.131817 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.113923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42490)
0807 22:49:31.114186 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.114211 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.114355 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.114795 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.114802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.115417 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.115695 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.115723 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.116867 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.116875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.118431 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.118452 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.119617 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.119625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.119917 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.120808 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.120843 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.123990 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.128736 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.128741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.128745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.129378 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.129886 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.129890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.129892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.130053 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.130645 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.130650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.130865 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.131019 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.131410 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:49:31.133049 12733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:31.134342 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.133716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59860)
0807 22:49:31.133963 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.134085 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 22:49:31.135630 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.135141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59874)
0807 22:49:31.135338 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.135444 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 22:49:31.135957 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.187268 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 1
I20260807 22:49:31.188632 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.188031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59884)
0807 22:49:31.188292 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.188410 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 22:49:31.189982 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.189544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59894)
0807 22:49:31.189744 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.189845 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 22:49:31.190264 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.191787 12815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 2
I20260807 22:49:31.193017 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.192510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59910)
0807 22:49:31.192726 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.192831 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 22:49:31.194361 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.193938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59920)
0807 22:49:31.194103 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.194193 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 22:49:31.194636 12815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.258544 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 2
I20260807 22:49:31.260162 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.259430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59928)
0807 22:49:31.259735 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.259892 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 22:49:31.261420 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.260933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59942)
0807 22:49:31.261117 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.261213 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:49:31.261806 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.285979 12815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 3
I20260807 22:49:31.287578 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.286935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59954)
0807 22:49:31.287185 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.287322 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 22:49:31.289129 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.288464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59956)
0807 22:49:31.288776 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.288915 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260807 22:49:31.289520 12815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.368095 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 3
I20260807 22:49:31.369771 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.369127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59968)
0807 22:49:31.369438 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.369556 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 22:49:31.371198 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.370776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59982)
0807 22:49:31.370980 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.371069 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 22:49:31.371438 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.432790 12815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 4
I20260807 22:49:31.434310 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.433590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:59996)
0807 22:49:31.433934 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.434075 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 22:49:31.435791 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.435316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60006)
0807 22:49:31.435504 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.435637 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 22:49:31.436125 12815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
W20260807 22:49:31.491457 12823 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:31.491842 12823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:31.491909 12823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:31.491995 12823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:31.500084 12823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:31.500288 12823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:31.500356 12823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:31.500422 12823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:31.500496 12823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:31.511150 12823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:44107,127.20.10.189:46733,127.20.10.188:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.10.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:31.512640 12823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:31.514098 12823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:31.527411 12830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:31.527385 12833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:31.527681 12831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:31.529484 12823 server_base.cc:1034] running on GCE node
I20260807 22:49:31.530757 12823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:31.532552 12823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:31.533842 12823 hybrid_clock.cc:648] HybridClock initialized: now 1786142971533788 us; error 37 us; skew 500 ppm
I20260807 22:49:31.534252 12823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142971, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:31.537976 12823 init.cc:377] Logged in from keytab as kudu/127.20.10.188@KRBTEST.COM (short username kudu)
I20260807 22:49:31.539878 12823 webserver.cc:460] Webserver started at http://127.20.10.188:37693/ using document root <none> and password file <none>
I20260807 22:49:31.540684 12823 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:31.540825 12823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:31.541217 12823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:31.544994 12823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data/instance:
uuid: "e4af497d4a7d42d0a0a4efca0e36ed15"
format_stamp: "Formatted at 2026-08-07 22:49:31 on dist-test-slave-csg5"
server_key: "4de6968a7dc30113318ae4a2edd7d865"
server_key_iv: "d38f4814ad7fc16e208768647a0345f8"
server_key_version: "encryptionkey@0"
I20260807 22:49:31.545970 12823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal/instance:
uuid: "e4af497d4a7d42d0a0a4efca0e36ed15"
format_stamp: "Formatted at 2026-08-07 22:49:31 on dist-test-slave-csg5"
server_key: "4de6968a7dc30113318ae4a2edd7d865"
server_key_iv: "d38f4814ad7fc16e208768647a0345f8"
server_key_version: "encryptionkey@0"
I20260807 22:49:31.548718 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 4
I20260807 22:49:31.549975 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.549493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60008)
0807 22:49:31.549720 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.549824 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 22:49:31.551342 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.550811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60024)
0807 22:49:31.551070 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.551176 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 22:49:31.551596 12733 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:36625: Network error: Client connection negotiation failed: client connection to 127.20.10.188:36625: connect: Connection refused (error 111)
I20260807 22:49:31.552924 12823 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260807 22:49:31.557631 12840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:31.559597 12823 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:49:31.559790 12823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "e4af497d4a7d42d0a0a4efca0e36ed15"
format_stamp: "Formatted at 2026-08-07 22:49:31 on dist-test-slave-csg5"
server_key: "4de6968a7dc30113318ae4a2edd7d865"
server_key_iv: "d38f4814ad7fc16e208768647a0345f8"
server_key_version: "encryptionkey@0"
I20260807 22:49:31.560009 12823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:31.581354 12823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:31.585958 12823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:31.586205 12823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:31.607321 12823 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.188:36625
I20260807 22:49:31.607421 12896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.188:36625 every 8 connection(s)
I20260807 22:49:31.608781 12823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
I20260807 22:49:31.610316 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 12823
I20260807 22:49:31.610531 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142560216906-20522-0/minicluster-data/master-2/wal/instance
I20260807 22:49:31.611196 20522 external_mini_cluster.cc:1428] Setting key 67ccbca057e92b391ba0ce88c7fdf24f
I20260807 22:49:31.614130 12897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:31.622226 12897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142971, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:31.637850 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.613333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.20.10.188:36625)
0807 22:49:31.614284 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.614300 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.614343 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:49:31.616065 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.616089 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.616604 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.617049 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.618632 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.620130 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.621577 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.622428 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.622662 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.627947 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.627955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.627966 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.628006 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.632285 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.633316 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.633328 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.633343 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.633593 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.634263 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.634272 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.634283 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.635217 (+ 934us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.635542 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.636003 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.636476 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":387,"threads_started":1}
I20260807 22:49:31.645045 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.625479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.20.10.190:44107)
0807 22:49:31.626090 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.626097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.626810 (+ 713us) server_negotiation.cc:407] Connection header received
0807 22:49:31.627049 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.627055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.627160 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.627290 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.628918 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.630331 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.632404 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.632927 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.633097 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.637479 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.637487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.637492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.637549 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.641103 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.642069 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.642075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.642078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.642274 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.642834 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.642841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.642844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.643255 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.643419 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.644361 (+ 942us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.644635 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":327,"threads_started":1}
I20260807 22:49:31.645071 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.625458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:44107 (local address 127.0.0.1:60558)
0807 22:49:31.626605 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.626695 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.626895 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.627443 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.627449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.628175 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.628673 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.628689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.630591 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.630596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.632071 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.632093 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.633308 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.633321 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.633598 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.634360 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.634387 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.637223 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.641294 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.641304 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.641309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.641877 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.642450 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.642455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.642458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.642687 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.643423 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.643432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.643743 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.644082 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.644540 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"thread_start_us":681,"threads_started":1}
I20260807 22:49:31.646847 12897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142971, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:31.661672 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.642042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.20.10.190:44107)
0807 22:49:31.642818 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.642823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.642846 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:49:31.642889 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.642894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.642989 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.643091 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.645100 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.646405 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.648129 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.648825 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.648961 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.654077 (+ 5116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.654083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.654087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.654126 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.657874 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.658822 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.658830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.658834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.659028 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.659617 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.659640 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.659647 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.660158 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.660499 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.660875 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.661157 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":333,"threads_started":1}
I20260807 22:49:31.664966 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.647624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:46733 (local address 127.0.0.1:42502)
0807 22:49:31.647907 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.647930 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.648067 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.649005 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.649011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.649625 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.649841 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.649857 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.651167 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.651174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.652520 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.652533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.653531 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.653539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.653773 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.654440 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.654467 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.656870 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.661807 (+ 4937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.661815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.661821 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.662351 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.662894 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.662900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.662903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.663083 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.663865 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.663870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.664082 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.664250 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.664583 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 22:49:31.665071 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.647760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.20.10.189:46733)
0807 22:49:31.648653 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.648661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.648679 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:49:31.648741 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.648747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.648837 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.648942 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.650100 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.650995 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.652693 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.653344 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.653459 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.657107 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.657114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.657121 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.657175 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.661597 (+ 4422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.662496 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.662502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.662506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.662737 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.663215 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.663220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.663224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.663621 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.663778 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.664346 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.664573 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":631,"threads_started":1}
I20260807 22:49:31.666594 12897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142971, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:31.684280 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.667746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.20.10.188:36625)
0807 22:49:31.668028 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.668040 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.668058 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:49:31.668100 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.668103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.668237 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.668319 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.669463 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.670555 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.672594 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.673259 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.673406 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.676991 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.676998 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.677003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.677049 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.680663 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.681752 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.681760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.681766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.681940 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.682389 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.682394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.682397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.682848 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.682965 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.683490 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.683839 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":59}
I20260807 22:49:31.684948 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.667544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60042)
0807 22:49:31.667742 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.667762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.667896 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.668380 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.668385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.668895 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.669068 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.669081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.670817 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.670822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.672429 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.672444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.673463 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.673468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.673661 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.674369 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.674397 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.676780 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.680869 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.680877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.680885 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.681587 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.682103 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.682107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.682109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.682253 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.682985 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.682990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.683189 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.683345 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.684198 (+ 853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":259}
I20260807 22:49:31.686479 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.663844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.20.10.189:46733)
0807 22:49:31.665667 (+ 1823us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.665677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.665709 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:49:31.665751 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.665755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.665814 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.665897 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.667977 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.669190 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.671172 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.672005 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.672159 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.678047 (+ 5888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.678054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.678057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.678094 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.681409 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.682643 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.682648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.682651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.682997 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.683464 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.683473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.683478 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.683924 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.684253 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.685674 (+ 1421us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.685997 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1690,"thread_start_us":1453,"threads_started":1}
I20260807 22:49:31.691052 12897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: Bootstrap starting.
I20260807 22:49:31.695128 12897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:31.695739 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:31.696581 12897 log.cc:824] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:31.700115 12897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: No bootstrap required, opened a new log
I20260807 22:49:31.699524 12707 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d10aa6566ca481bbda8f273a6867b39"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60572
I20260807 22:49:31.708146 12897 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.708549 12897 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4af497d4a7d42d0a0a4efca0e36ed15, State: Initialized, Role: FOLLOWER
I20260807 22:49:31.709090 12897 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.710306 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:31.710755 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:31.711510 12897 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:49:31.722187 12919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:49:31.722349 12919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:49:31.719547 12823 master_runner.cc:186] Error getting master registration for 127.20.10.190:44107: OK, instance_id { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" instance_seqno: 1786142970390970 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:49:31.731897 12823 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:49:31.735392 12815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 5
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
I20260807 22:49:31.752549 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.736130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60048)
0807 22:49:31.736376 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.736401 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.736541 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.736964 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.736969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.737534 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.737762 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.737776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.739298 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.739305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.740826 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.740846 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.742000 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.742007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.742449 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.743238 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.743289 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.746050 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.749776 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.749782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.749786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.750249 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.750695 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.750700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.750703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.750857 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.751485 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.751492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.751661 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.751833 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.752150 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 22:49:31.752555 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.736268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.20.10.188:36625)
0807 22:49:31.736486 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.736491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.736518 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:49:31.736693 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.736698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.736788 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.736899 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.737939 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.739137 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.740995 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.741792 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.741928 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.746307 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.746315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.746351 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.746396 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.749586 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.750395 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.750402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.750406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.750542 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.750974 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.750980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.750983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.751358 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.751538 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.751937 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.752171 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:49:31.759092 12815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: Bootstrap starting.
I20260807 22:49:31.763931 12815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:31.765496 12815 log.cc:824] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:31.769801 12815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: No bootstrap required, opened a new log
I20260807 22:49:31.782073 12815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.782585 12815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c50c5e7f0e04c978370ad61d2e092a2, State: Initialized, Role: FOLLOWER
I20260807 22:49:31.783205 12815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.784390 12922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:31.784924 12922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:31.785880 12815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:31.793984 12740 master_runner.cc:186] Error getting master registration for 127.20.10.190:44107: OK, instance_id { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" instance_seqno: 1786142970390970 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:49:31.805686 12740 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 22:49:31.808849 12933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:49:31.809017 12933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:49:31.828874 12733 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } attempt: 5
Aug 07 22:49:31 dist-test-slave-csg5 krb5kdc[12647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142970, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
W20260807 22:49:31.838538 12847 tablet.cc:2369] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:31.847225 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.829811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.20.10.188:36625)
0807 22:49:31.830078 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.830086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.830341 (+ 255us) server_negotiation.cc:407] Connection header received
0807 22:49:31.830565 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.830571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.830698 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.830794 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.831652 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.832880 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.834572 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.835349 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.835485 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.840083 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.840092 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.840098 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.840146 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.844215 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.844993 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.845000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.845006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.845202 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.845632 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.845639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.845646 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.846011 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.846159 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.846563 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.846839 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 22:49:31.847249 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.829732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:36625 (local address 127.0.0.1:60060)
0807 22:49:31.830242 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:31.830264 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:31.830405 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:31.830838 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:31.830843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:31.831298 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:31.831472 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.831482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.833014 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.833021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.834412 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:31.834428 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.835516 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.835522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:31.835753 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.836421 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:31.836440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:31.839850 (+ 3410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:31.844374 (+ 4524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.844380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.844386 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.844861 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.845331 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:31.845334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:31.845337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:31.845494 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:31.846148 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:31.846153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:31.846304 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 22:49:31.846455 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:49:31.846815 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383}
I20260807 22:49:31.852450 12733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: Bootstrap starting.
I20260807 22:49:31.857026 12733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:31.858539 12733 log.cc:824] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:31.862622 12733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: No bootstrap required, opened a new log
I20260807 22:49:31.870820 12733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.871299 12733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d10aa6566ca481bbda8f273a6867b39, State: Initialized, Role: FOLLOWER
I20260807 22:49:31.871872 12733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.873243 12936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:31.874019 12936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:31.874910 12733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:31.874992 12707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:49:31.875181 12707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 0 FOLLOWER]: Advancing to term 1
W20260807 22:49:31.878813 12684 tablet.cc:2369] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:31.879397 12707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.882578 12787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d10aa6566ca481bbda8f273a6867b39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7c50c5e7f0e04c978370ad61d2e092a2"
I20260807 22:49:31.883355 12787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:31.889391 12787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d10aa6566ca481bbda8f273a6867b39 in term 1.
I20260807 22:49:31.891295 12871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d10aa6566ca481bbda8f273a6867b39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15"
I20260807 22:49:31.892433 12871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:31.897712 12871 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d10aa6566ca481bbda8f273a6867b39 in term 1.
I20260807 22:49:31.899395 12677 leader_election.cc:302] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d10aa6566ca481bbda8f273a6867b39, e4af497d4a7d42d0a0a4efca0e36ed15; no voters:
I20260807 22:49:31.901031 12707 leader_election.cc:288] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [CANDIDATE]: Term 1 election: Requested vote from peers 7c50c5e7f0e04c978370ad61d2e092a2 (127.20.10.189:46733), e4af497d4a7d42d0a0a4efca0e36ed15 (127.20.10.188:36625)
I20260807 22:49:31.908856 12936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:31.919552 12936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [term 1 LEADER]: Becoming Leader. State: Replica: 3d10aa6566ca481bbda8f273a6867b39, State: Running, Role: LEADER
I20260807 22:49:31.920622 12936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } }
I20260807 22:49:31.933283 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d10aa6566ca481bbda8f273a6867b39. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:31.934303 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:31.937413 12953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260807 22:49:31.975440 12764 tablet.cc:2369] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:31.975069 12953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:49:31.982342 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.945642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.20.10.190:44107)
0807 22:49:31.945866 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.945874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.955261 (+ 9387us) server_negotiation.cc:407] Connection header received
0807 22:49:31.955603 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.955609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.955706 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.955783 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.958402 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.959628 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.961694 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.962641 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.962793 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.974003 (+ 11210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.974012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.974017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.974068 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.978150 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.979089 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.979095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.979143 (+ 48us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.979328 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.979853 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.979859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.979864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.980344 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.980624 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.981693 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.981946 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 22:49:31.985451 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.944334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.20.10.189:46733)
0807 22:49:31.944590 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.944597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.946498 (+ 1901us) server_negotiation.cc:407] Connection header received
0807 22:49:31.947656 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.947666 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.947817 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.947912 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.953461 (+ 5549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.954675 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.962251 (+ 7576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.963032 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.963153 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.970337 (+ 7184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.970348 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.970354 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.970411 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:31.974606 (+ 4195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.980272 (+ 5666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.980281 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.980286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.980473 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:31.981309 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:31.981314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:31.981317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:31.981776 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:31.982082 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:31.984781 (+ 2699us) server_negotiation.cc:299] Negotiation successful
0807 22:49:31.985073 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 22:49:31.988802 12960 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:49:31.996764 12871 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [term 1 FOLLOWER]: Refusing update from remote peer 3d10aa6566ca481bbda8f273a6867b39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:31.998009 12936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:32.007658 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:31.955953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.20.10.188:36625)
0807 22:49:31.956229 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:31.956235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:31.958125 (+ 1890us) server_negotiation.cc:407] Connection header received
0807 22:49:31.959151 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:31.959159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:31.959258 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:31.959352 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:31.975501 (+ 16149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.976639 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.983081 (+ 6442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:31.983858 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:31.983969 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:31.998886 (+ 14917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:31.998894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:31.998901 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:31.998955 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:32.003464 (+ 4509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:32.004649 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:32.004656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:32.004660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:32.004832 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:32.005409 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:32.005415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:32.005419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:32.005863 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:32.006182 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:32.006861 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 22:49:32.007180 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 22:49:32.008575 12787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [term 1 FOLLOWER]: Refusing update from remote peer 3d10aa6566ca481bbda8f273a6867b39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:32.012315 12936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:32.041208 12922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d10aa6566ca481bbda8f273a6867b39. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.041787 12922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:32.044979 12961 mvcc.cc:205] Tried to move back new op lower bound from 7316041613274742784 to 7316041613016236032. Current Snapshot: MvccSnapshot[applied={T|T < 7316041613274742784}]
I20260807 22:49:32.063598 12922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.064101 12922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:32.069600 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.070103 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:32.071285 12953 catalog_manager.cc:1347] Generated new cluster ID: 8d25337874704e30b9f9a1c7e144ba11
I20260807 22:49:32.071419 12953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:32.093999 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d10aa6566ca481bbda8f273a6867b39. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.094398 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:32.108803 12936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.109233 12936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:32.124235 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d10aa6566ca481bbda8f273a6867b39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d10aa6566ca481bbda8f273a6867b39" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 44107 } } peers { permanent_uuid: "7c50c5e7f0e04c978370ad61d2e092a2" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 46733 } } peers { permanent_uuid: "e4af497d4a7d42d0a0a4efca0e36ed15" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 36625 } } }
I20260807 22:49:32.124594 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:32.146552 12953 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:32.148699 12953 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:32.184176 12953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d10aa6566ca481bbda8f273a6867b39: Generated new TSK 0
I20260807 22:49:32.185454 12953 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:49:32.350303 12903 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.336199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.20.10.190:44107)
0807 22:49:32.336459 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.336472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.348070 (+ 11598us) server_negotiation.cc:407] Connection header received
0807 22:49:32.348333 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.348343 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.348460 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.348580 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.349978 (+ 1398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60594: BlockingRecv error: recv got EOF from 127.0.0.1:60594 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 22:49:32.351002 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.338288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.20.10.189:46733)
0807 22:49:32.338478 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.338486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.348859 (+ 10373us) server_negotiation.cc:407] Connection header received
0807 22:49:32.349174 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.349180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.349308 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.349396 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.350738 (+ 1342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42524: BlockingRecv error: recv got EOF from 127.0.0.1:42524 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 22:49:32.353044 12900 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.348554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.20.10.188:36625)
0807 22:49:32.348769 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.348778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.351176 (+ 2398us) server_negotiation.cc:407] Connection header received
0807 22:49:32.351468 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.351475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.351571 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.351658 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.352742 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60076: BlockingRecv error: recv got EOF from 127.0.0.1:60076 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 22:49:32.380781 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.377298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.20.10.189:46733)
0807 22:49:32.377503 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.377509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.379070 (+ 1561us) server_negotiation.cc:407] Connection header received
0807 22:49:32.379244 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.379252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.379428 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.379555 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.380529 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42538: BlockingRecv error: recv got EOF from 127.0.0.1:42538 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 22:49:32.392452 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.390698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.20.10.189:46733)
0807 22:49:32.390917 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.390923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.390948 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:49:32.391045 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.391051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.391139 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.391262 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.392175 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42542: BlockingRecv error: recv got EOF from 127.0.0.1:42542 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 22:49:32.433174 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.428168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.20.10.189:46733)
0807 22:49:32.428383 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.428391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.431381 (+ 2990us) server_negotiation.cc:407] Connection header received
0807 22:49:32.431614 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.431622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.431769 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.431911 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.432908 (+ 997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42544: BlockingRecv error: recv got EOF from 127.0.0.1:42544 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 22:49:32.444978 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.442398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.20.10.189:46733)
0807 22:49:32.442611 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.442620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.443206 (+ 586us) server_negotiation.cc:407] Connection header received
0807 22:49:32.443523 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.443530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.443620 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.443727 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.444702 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42554: BlockingRecv error: recv got EOF from 127.0.0.1:42554 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 22:49:32.459329 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.457024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.20.10.189:46733)
0807 22:49:32.457259 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.457265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.457661 (+ 396us) server_negotiation.cc:407] Connection header received
0807 22:49:32.457876 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.457880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.457975 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.458063 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.459097 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42566: BlockingRecv error: recv got EOF from 127.0.0.1:42566 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 22:49:32.481932 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.479603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.20.10.189:46733)
0807 22:49:32.479891 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.479898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.480164 (+ 266us) server_negotiation.cc:407] Connection header received
0807 22:49:32.480401 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.480405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.480487 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.480575 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.481587 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42574: BlockingRecv error: recv got EOF from 127.0.0.1:42574 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260807 22:49:32.520328 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.518022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.20.10.189:46733)
0807 22:49:32.518292 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.518299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.518755 (+ 456us) server_negotiation.cc:407] Connection header received
0807 22:49:32.518997 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.519004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.519113 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.519267 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.520059 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42590: BlockingRecv error: recv got EOF from 127.0.0.1:42590 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 22:49:32.591246 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.588669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.20.10.189:46733)
0807 22:49:32.588924 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.588933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.589509 (+ 576us) server_negotiation.cc:407] Connection header received
0807 22:49:32.589817 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.589823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.589942 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.590067 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.590944 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42598: BlockingRecv error: recv got EOF from 127.0.0.1:42598 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 22:49:32.725240 12919 catalog_manager.cc:1259] Loaded cluster ID: 8d25337874704e30b9f9a1c7e144ba11
I20260807 22:49:32.725399 12919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: loading cluster ID for follower catalog manager: success
W20260807 22:49:32.726472 12905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:32.723569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.20.10.189:46733)
0807 22:49:32.723815 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.723832 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.724672 (+ 840us) server_negotiation.cc:407] Connection header received
0807 22:49:32.724976 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.724982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.725075 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.725191 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:32.726212 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42606: BlockingRecv error: recv got EOF from 127.0.0.1:42606 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 22:49:32.728957 12919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: acquiring CA information for follower catalog manager: success
I20260807 22:49:32.731249 12919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4af497d4a7d42d0a0a4efca0e36ed15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:49:32.811355 12933 catalog_manager.cc:1259] Loaded cluster ID: 8d25337874704e30b9f9a1c7e144ba11
I20260807 22:49:32.811466 12933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: loading cluster ID for follower catalog manager: success
I20260807 22:49:32.814273 12933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: acquiring CA information for follower catalog manager: success
I20260807 22:49:32.816303 12933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7c50c5e7f0e04c978370ad61d2e092a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:49:32.994385 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:32.988089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42608 (local address 127.20.10.189:46733)
0807 22:49:32.988285 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:32.988292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:32.988638 (+ 346us) server_negotiation.cc:407] Connection header received
0807 22:49:32.988835 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:32.988839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:32.989011 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:32.989140 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:32.989860 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:32.990641 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:32.992159 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:32.992566 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:32.992667 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:32.993215 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:32.993742 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:32.993911 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:32.994040 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 22:49:32.994122 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:49:33.019241 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:33.005161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.20.10.188:36625)
0807 22:49:33.005802 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:33.005812 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:33.012132 (+ 6320us) server_negotiation.cc:407] Connection header received
0807 22:49:33.012471 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:33.012476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:33.012821 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:33.013012 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:33.013582 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.014629 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.016493 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.016963 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.017079 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:33.017759 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:33.018329 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:33.018446 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:33.018763 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:49:33.018878 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":286,"threads_started":1}
I20260807 22:49:33.037173 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:33.029322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.20.10.188:36625)
0807 22:49:33.029705 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:33.029715 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:33.030338 (+ 623us) server_negotiation.cc:407] Connection header received
0807 22:49:33.030449 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:33.030454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:33.030536 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:33.030641 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:33.031287 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.032329 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.034856 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.035561 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.035770 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:33.036360 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:33.036498 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:33.036592 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:33.036798 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 22:49:33.036885 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260807 22:49:33.037182 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:33.027615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.20.10.189:46733)
0807 22:49:33.027928 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:33.027938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:33.029791 (+ 1853us) server_negotiation.cc:407] Connection header received
0807 22:49:33.030638 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:33.030645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:33.030786 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:33.030890 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:33.031685 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.032676 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.034699 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.035388 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.035535 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:33.036159 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:33.036370 (+ 211us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:33.036453 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:33.036745 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 22:49:33.036845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 22:49:33.037949 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:33.027490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.20.10.190:44107)
0807 22:49:33.030199 (+ 2709us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:33.030205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:33.030222 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:49:33.030371 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:33.030377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:33.030615 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:33.030844 (+ 229us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:33.031772 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.032627 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.035059 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:33.035547 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:33.035692 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:33.036288 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:33.036820 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:33.037119 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:33.037627 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 22:49:33.037723 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2601,"thread_start_us":2491,"threads_started":1}
I20260807 22:49:33.044312 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12660
I20260807 22:49:33.045498 12728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:33.161734 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12660
I20260807 22:49:33.191939 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12740
I20260807 22:49:33.193356 12809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:33.313843 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12740
I20260807 22:49:33.342283 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 12823
I20260807 22:49:33.343581 12892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:33.462188 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 12823
2026-08-07T22:49:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5291 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:49:33 dist-test-slave-csg5 krb5kdc[13056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:49:33 dist-test-slave-csg5 krb5kdc[13056](info): set up 2 sockets
Aug 07 22:49:33 dist-test-slave-csg5 krb5kdc[13056](info): commencing operation
krb5kdc: starting...
W20260807 22:49:34.704551 20522 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/test-user.keytab.
Aug 07 22:49:34 dist-test-slave-csg5 krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.10.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
Entry for principal kudu/127.20.10.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab.
I20260807 22:49:35.107995 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39309
--webserver_interface=127.20.10.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:49:35.509900 13070 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:35.510293 13070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:35.510404 13070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:35.510545 13070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:35.518749 13070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:35.518865 13070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:35.518967 13070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:35.519043 13070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:35.519141 13070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:35.530025 13070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.190.keytab
--principal=kudu/127.20.10.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.190:39309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.10.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:35.531601 13070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:35.533041 13070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:35.547690 13075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:35.547571 13079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:35.547606 13076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:35.549098 13070 server_base.cc:1034] running on GCE node
I20260807 22:49:35.550117 13070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:35.551918 13070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:35.553237 13070 hybrid_clock.cc:648] HybridClock initialized: now 1786142975553136 us; error 71 us; skew 500 ppm
I20260807 22:49:35.553696 13070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:35 dist-test-slave-csg5 krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142975, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:35.557458 13070 init.cc:377] Logged in from keytab as kudu/127.20.10.190@KRBTEST.COM (short username kudu)
I20260807 22:49:35.559182 13070 webserver.cc:460] Webserver started at http://127.20.10.190:37381/ using document root <none> and password file <none>
I20260807 22:49:35.559926 13070 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:35.560020 13070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:35.560303 13070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:35.563647 13070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data/instance:
uuid: "a0effe12dfe94e69a6c40eddf1c22cd4"
format_stamp: "Formatted at 2026-08-07 22:49:35 on dist-test-slave-csg5"
server_key: "9e95f79dbdd6a050180ea828a88c561d"
server_key_iv: "d231d5b47d8961bd27a4178b547cac0c"
server_key_version: "encryptionkey@0"
I20260807 22:49:35.564492 13070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance:
uuid: "a0effe12dfe94e69a6c40eddf1c22cd4"
format_stamp: "Formatted at 2026-08-07 22:49:35 on dist-test-slave-csg5"
server_key: "9e95f79dbdd6a050180ea828a88c561d"
server_key_iv: "d231d5b47d8961bd27a4178b547cac0c"
server_key_version: "encryptionkey@0"
I20260807 22:49:35.570428 13070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 22:49:35.574770 13086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:35.576380 13070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:49:35.576551 13070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal
uuid: "a0effe12dfe94e69a6c40eddf1c22cd4"
format_stamp: "Formatted at 2026-08-07 22:49:35 on dist-test-slave-csg5"
server_key: "9e95f79dbdd6a050180ea828a88c561d"
server_key_iv: "d231d5b47d8961bd27a4178b547cac0c"
server_key_version: "encryptionkey@0"
I20260807 22:49:35.576766 13070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:35.599411 13070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:35.606348 13070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:35.606768 13070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:35.631022 13070 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.190:39309
I20260807 22:49:35.631044 13143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.190:39309 every 8 connection(s)
I20260807 22:49:35.632509 13070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/data/info.pb
I20260807 22:49:35.636664 13144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:35.639047 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 13070
I20260807 22:49:35.639254 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-0/wal/instance
I20260807 22:49:35.639849 20522 external_mini_cluster.cc:1428] Setting key b4bfddb797fc8a7a3224820282a67c37
I20260807 22:49:35.646968 13144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:35 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142975, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:35.675163 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.652889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54242 (local address 127.20.10.190:39309)
0807 22:49:35.653727 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:35.653738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:35.653801 (+ 63us) server_negotiation.cc:407] Connection header received
0807 22:49:35.655240 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:35.655250 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:35.655602 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:35.655871 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:35.657174 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:35.658431 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:35.660792 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:35.661508 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:35.661679 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:35.666503 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:35.666512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:35.666527 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:35.666562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:35.669876 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:35.670777 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:35.670784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:35.670787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:35.670950 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:35.671427 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:35.671437 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:35.671440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:35.672031 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:35.672294 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:35.672883 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 22:49:35.673366 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":591,"thread_start_us":407,"threads_started":1}
I20260807 22:49:35.675185 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.651448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39309 (local address 127.0.0.1:54242)
0807 22:49:35.652366 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.652419 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:35.654808 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:35.655919 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:35.655925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:35.656553 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:35.656929 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:35.656947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:35.658610 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:35.658618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:35.660615 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:35.660636 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:35.661797 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:35.661807 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:35.662056 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:35.663036 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:35.663076 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:35.666217 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:35.670067 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:35.670082 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:35.670097 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:35.670571 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:35.671084 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:35.671093 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:35.671098 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:35.671295 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:35.672185 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:35.672193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:35.672450 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 22:49:35.672728 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 22:49:35.673549 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":210,"thread_start_us":398,"threads_started":1}
WARNING: no policy specified for kudu/127.20.10.189@KRBTEST.COM; defaulting to no policy
I20260807 22:49:35.678958 13144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:35.680387 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.679899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34452)
0807 22:49:35.680096 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.680218 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 22:49:35.680773 13089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.189@KRBTEST.COM" created.
I20260807 22:49:35.682937 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.682444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34468)
0807 22:49:35.682622 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.682779 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:49:35.683226 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:32775: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
Entry for principal kudu/127.20.10.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab.
I20260807 22:49:35.728716 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:32775
--webserver_interface=127.20.10.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:49:35.732616 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } attempt: 1
I20260807 22:49:35.733878 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.733400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34478)
0807 22:49:35.733615 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.733731 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 22:49:35.735251 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.734853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34480)
0807 22:49:35.735021 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.735093 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:49:35.735460 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:32775: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
I20260807 22:49:35.803804 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } attempt: 2
I20260807 22:49:35.805157 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.804585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34486)
0807 22:49:35.804890 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.804986 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:49:35.806294 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.805891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34492)
0807 22:49:35.806059 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.806146 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:49:35.806565 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:32775: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
I20260807 22:49:35.897889 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } attempt: 3
I20260807 22:49:35.899277 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.898798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34502)
0807 22:49:35.899017 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.899134 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:49:35.900429 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:35.900019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34510)
0807 22:49:35.900191 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:35.900265 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 22:49:35.900748 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:32775: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
I20260807 22:49:36.044099 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } attempt: 4
I20260807 22:49:36.045655 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.044947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34516)
0807 22:49:36.045412 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.045508 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305}
I20260807 22:49:36.046919 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.046486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34524)
0807 22:49:36.046628 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.046785 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:49:36.047286 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.189:32775: Network error: Client connection negotiation failed: client connection to 127.20.10.189:32775: connect: Connection refused (error 111)
W20260807 22:49:36.115407 13152 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:36.115845 13152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:36.115952 13152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:36.116143 13152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:36.125013 13152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:36.125173 13152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:36.125245 13152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:36.125298 13152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:36.125409 13152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:36.138459 13152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.189.keytab
--principal=kudu/127.20.10.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.189:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.10.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:36.140100 13152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:36.141896 13152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:36.157050 13158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:36.157002 13160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:36.157871 13152 server_base.cc:1034] running on GCE node
W20260807 22:49:36.158016 13157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:36.159168 13152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:36.160786 13152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:36.162031 13152 hybrid_clock.cc:648] HybridClock initialized: now 1786142976161941 us; error 64 us; skew 500 ppm
I20260807 22:49:36.162387 13152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:36.166265 13152 init.cc:377] Logged in from keytab as kudu/127.20.10.189@KRBTEST.COM (short username kudu)
I20260807 22:49:36.168113 13152 webserver.cc:460] Webserver started at http://127.20.10.189:46517/ using document root <none> and password file <none>
I20260807 22:49:36.168855 13152 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:36.168949 13152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:36.169250 13152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:36.172592 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data/instance:
uuid: "20c0590b98aa4ac79a56bd72f276bbc3"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "fce5fb04a26d1b5238fb8f597fdc9b6f"
server_key_iv: "9e87b3f3519d6989e3011d70c6f8864a"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.173429 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal/instance:
uuid: "20c0590b98aa4ac79a56bd72f276bbc3"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "fce5fb04a26d1b5238fb8f597fdc9b6f"
server_key_iv: "9e87b3f3519d6989e3011d70c6f8864a"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.180438 13152 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260807 22:49:36.185051 13168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:36.186965 13152 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:49:36.187155 13152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal
uuid: "20c0590b98aa4ac79a56bd72f276bbc3"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "fce5fb04a26d1b5238fb8f597fdc9b6f"
server_key_iv: "9e87b3f3519d6989e3011d70c6f8864a"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.187381 13152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:36.201040 13152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:36.206413 13152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:36.206696 13152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:36.229032 13152 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.189:32775
I20260807 22:49:36.229053 13222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.189:32775 every 8 connection(s)
I20260807 22:49:36.230563 13152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/data/info.pb
I20260807 22:49:36.234892 13223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:36.239234 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 13152
I20260807 22:49:36.239480 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-1/wal/instance
I20260807 22:49:36.240073 20522 external_mini_cluster.cc:1428] Setting key d6cfd12e8847317812d1a57355f6b145
I20260807 22:49:36.243860 13223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:36.270072 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.247730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.20.10.190:39309)
0807 22:49:36.248349 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.248358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.249623 (+ 1265us) server_negotiation.cc:407] Connection header received
0807 22:49:36.252091 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.252101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.252206 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.252355 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.254555 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.255729 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.257583 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.258285 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.258473 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.263394 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.263400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.263403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.263454 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.266409 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.267414 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.267423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.267426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.267594 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.268098 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.268101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.268104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.268427 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.268596 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.269403 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.269654 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":283,"threads_started":1}
I20260807 22:49:36.271298 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.248405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39309 (local address 127.0.0.1:54252)
0807 22:49:36.249396 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.249465 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.251731 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.252695 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.252705 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.253523 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.254255 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.254278 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.255957 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.255961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.257387 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.257403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.258699 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.258710 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.259094 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.259940 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.259986 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.263048 (+ 3062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.266614 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.266624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.266644 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.267240 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.267721 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.267726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.267730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.267957 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.268573 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.268583 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.269009 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.269297 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.269895 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":442,"threads_started":1}
I20260807 22:49:36.273409 13223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.10.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.10.188@KRBTEST.COM" created.
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:36.292805 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.274464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34532)
0807 22:49:36.274911 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.274939 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.275107 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.276445 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.276453 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.277127 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.277375 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.277390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.279135 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.279140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.280422 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.280439 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.281763 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.281772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.282074 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.282997 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.283018 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.285923 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.289988 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.290002 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.290010 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.290405 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.290916 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.290922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.290928 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.291069 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.291749 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.291753 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.291903 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.292053 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.292386 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260807 22:49:36.292924 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.274708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.20.10.189:32775)
0807 22:49:36.275781 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.275789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.275823 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:49:36.275880 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.275887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.276134 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.276386 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.277600 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.278938 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.280608 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.281555 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.281680 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.286211 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.286237 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.286255 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.286298 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.289794 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.290571 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.290578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.290581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.290773 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.291188 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.291195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.291198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.291588 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.291826 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.292236 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.292517 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":717,"thread_start_us":463,"threads_started":1}
I20260807 22:49:36.296103 13223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:36.297538 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.296932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39282)
0807 22:49:36.297163 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.297344 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 22:49:36.297852 13169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.299897 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.299453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39298)
0807 22:49:36.299671 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.299766 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:49:36.300415 13223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
Entry for principal kudu/127.20.10.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab.
I20260807 22:49:36.331209 20522 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
/tmp/dist-test-taskKr6fn6/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:40213
--webserver_interface=127.20.10.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.10.148:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:49:36.346606 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } attempt: 5
I20260807 22:49:36.351577 13223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 1
I20260807 22:49:36.353353 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.352370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39314)
0807 22:49:36.352998 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.353137 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396}
I20260807 22:49:36.355185 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.354582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39324)
0807 22:49:36.354898 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.355003 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260807 22:49:36.355563 13223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142975, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:36.368248 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.347774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34540)
0807 22:49:36.347996 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.348023 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.348357 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.350189 (+ 1832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.350197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.350908 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.351274 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.351293 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.353782 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.353792 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.355703 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.355729 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.357102 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.357107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.357278 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.358082 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.358103 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.360381 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.365426 (+ 5045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.365431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.365434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.365873 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.366423 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.366426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.366429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.366572 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.367421 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.367425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.367573 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.367685 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.367965 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 22:49:36.368744 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.347897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.20.10.189:32775)
0807 22:49:36.348189 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.348197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.348244 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:49:36.348578 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.348587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.348721 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.348852 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.351517 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.353504 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.355945 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.356901 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.357106 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.360638 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.360648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.360656 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.360713 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.365169 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.366052 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.366061 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.366068 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.366248 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.366738 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.366747 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.366754 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.367264 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.367463 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.367886 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.368190 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 22:49:36.370035 13144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:49:36.371368 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.370903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39340)
0807 22:49:36.371096 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.371189 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:49:36.372419 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.372050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39356)
0807 22:49:36.372216 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.372312 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:49:36.372752 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.423834 13223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 2
I20260807 22:49:36.424042 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 1
I20260807 22:49:36.425535 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.424910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39384)
0807 22:49:36.425256 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.425343 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260807 22:49:36.425536 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.424921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39370)
0807 22:49:36.425188 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.425318 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 22:49:36.426895 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.426380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39400)
0807 22:49:36.426577 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.426734 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 22:49:36.426898 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.426461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39410)
0807 22:49:36.426602 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.426719 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:49:36.427163 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
W20260807 22:49:36.427313 13223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.495427 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 2
I20260807 22:49:36.496745 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.496212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39426)
0807 22:49:36.496465 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.496567 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 22:49:36.498010 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.497574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39430)
0807 22:49:36.497817 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.497904 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260807 22:49:36.498292 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.518595 13223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 3
I20260807 22:49:36.520078 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.519451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39436)
0807 22:49:36.519720 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.519838 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 22:49:36.521487 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.520886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39444)
0807 22:49:36.521161 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.521299 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260807 22:49:36.521852 13223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.604557 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 3
I20260807 22:49:36.606168 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.605511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39456)
0807 22:49:36.605840 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.605961 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260807 22:49:36.607699 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.607152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39472)
0807 22:49:36.607444 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.607547 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260807 22:49:36.608052 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.665165 13223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 4
I20260807 22:49:36.666703 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.666023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39482)
0807 22:49:36.666319 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.666443 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:49:36.668010 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.667543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39488)
0807 22:49:36.667748 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.667852 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:49:36.668383 13223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
W20260807 22:49:36.745189 13232 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:49:36.745638 13232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:49:36.745769 13232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:49:36.745910 13232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:49:36.753867 13232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:49:36.754024 13232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:49:36.754153 13232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:49:36.754264 13232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:49:36.754377 13232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:49:36.766187 13232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.10.148:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.10.190:39309,127.20.10.189:32775,127.20.10.188:40213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/krb5kdc/kudu_127.20.10.188.keytab
--principal=kudu/127.20.10.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.10.188:40213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.10.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 21:50:02 UTC on ccycloud.quasar-bxybku.root.comops.site
build id 4573649
ASAN enabled
I20260807 22:49:36.767982 13232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:49:36.769711 13232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:49:36.784387 13237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:36.784492 13240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:49:36.784154 13238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:49:36.785276 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 4
I20260807 22:49:36.785487 13232 server_base.cc:1034] running on GCE node
I20260807 22:49:36.786422 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.785946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39500)
0807 22:49:36.786162 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.786267 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 22:49:36.786418 13232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:49:36.787647 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.787242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39504)
0807 22:49:36.787423 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.787519 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 22:49:36.787911 13144 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.10.188:40213: Network error: Client connection negotiation failed: client connection to 127.20.10.188:40213: connect: Connection refused (error 111)
I20260807 22:49:36.788175 13232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:49:36.789470 13232 hybrid_clock.cc:648] HybridClock initialized: now 1786142976789381 us; error 54 us; skew 500 ppm
I20260807 22:49:36.789853 13232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:49:36.793933 13232 init.cc:377] Logged in from keytab as kudu/127.20.10.188@KRBTEST.COM (short username kudu)
I20260807 22:49:36.795939 13232 webserver.cc:460] Webserver started at http://127.20.10.188:37651/ using document root <none> and password file <none>
I20260807 22:49:36.796756 13232 fs_manager.cc:359] Metadata directory not provided
I20260807 22:49:36.796929 13232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:49:36.797312 13232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:49:36.800779 13232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data/instance:
uuid: "f947ea833b58493fb6febc56db0093e4"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "764f9594be6f376ff00f92a56054359b"
server_key_iv: "b100899d3a4763201cd8db1e8f317858"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.801677 13232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal/instance:
uuid: "f947ea833b58493fb6febc56db0093e4"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "764f9594be6f376ff00f92a56054359b"
server_key_iv: "b100899d3a4763201cd8db1e8f317858"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.808003 13232 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 22:49:36.812868 13247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:49:36.814813 13232 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 22:49:36.814990 13232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data,/tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal
uuid: "f947ea833b58493fb6febc56db0093e4"
format_stamp: "Formatted at 2026-08-07 22:49:36 on dist-test-slave-csg5"
server_key: "764f9594be6f376ff00f92a56054359b"
server_key_iv: "b100899d3a4763201cd8db1e8f317858"
server_key_version: "encryptionkey@0"
I20260807 22:49:36.815209 13232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:49:36.841379 13232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:49:36.846243 13232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:49:36.846500 13232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:49:36.867843 13232 rpc_server.cc:307] RPC server started. Bound to: 127.20.10.188:40213
I20260807 22:49:36.867863 13302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.10.188:40213 every 8 connection(s)
I20260807 22:49:36.869326 13232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/data/info.pb
I20260807 22:49:36.870448 20522 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu as pid 13232
I20260807 22:49:36.870695 20522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKr6fn6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142560216906-20522-0/minicluster-data/master-2/wal/instance
I20260807 22:49:36.871286 20522 external_mini_cluster.cc:1428] Setting key 5c65bfbe94451d45da25b88f4a7e1fb1
I20260807 22:49:36.874537 13303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:49:36.884680 13303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
I20260807 22:49:36.900619 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.873341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.20.10.188:40213)
0807 22:49:36.874196 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.874212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.874263 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:49:36.876659 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.876691 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.877247 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.877786 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.879478 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.881835 (+ 2357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.883481 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.884614 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.884839 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.890517 (+ 5678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.890525 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.890537 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.890592 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.895328 (+ 4736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.896283 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.896291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.896300 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.896535 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.897043 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.897048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.897050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.897712 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.898156 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.898758 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.899263 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":356,"threads_started":1}
I20260807 22:49:36.907595 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.888180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.190:39309 (local address 127.0.0.1:54260)
0807 22:49:36.889234 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.889298 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.889494 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.889974 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.889983 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.890934 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.891351 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.891363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.892677 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.892682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.894916 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.894940 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.896085 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.896094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.896340 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.897153 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.897173 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.900003 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.904170 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.904186 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.904192 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.904920 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.905471 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.905479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.905482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.905681 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.906554 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.906561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.906828 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.906990 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.907287 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":556,"threads_started":1}
I20260807 22:49:36.907986 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.888190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.20.10.190:39309)
0807 22:49:36.888834 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.888841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.889413 (+ 572us) server_negotiation.cc:407] Connection header received
0807 22:49:36.889647 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.889651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.889742 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.889843 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.891523 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.892482 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.895262 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.895808 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.896018 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.900268 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.900275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.900279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.900328 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.904000 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.905074 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.905093 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.905102 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.905315 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.905855 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.905860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.905863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.906376 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.906565 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.907214 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.907493 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":292,"threads_started":1}
I20260807 22:49:36.909668 13303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.190@KRBTEST.COM
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:36.924407 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.904822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.20.10.190:39309)
0807 22:49:36.905679 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.905685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.905711 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:49:36.905767 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.905773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.905856 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.905942 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.908578 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.909703 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.911656 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.912154 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.912312 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.917326 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.917333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.917337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.917404 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.921342 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.922138 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.922144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.922148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.922312 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.922776 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.922781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.922785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.923097 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.923414 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.923742 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.923994 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":384,"threads_started":1}
I20260807 22:49:36.925536 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.910528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.189:32775 (local address 127.0.0.1:34542)
0807 22:49:36.910742 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.910763 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.910864 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.911576 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.911583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.912065 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.912260 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.912270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.913560 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.913567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.914496 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.914509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.915318 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.915324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.915511 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.916206 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.916226 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.918985 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.922823 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.922829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.922833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.923267 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.923701 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.923707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.923711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.923874 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.924505 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.924510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.924695 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.924843 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.925174 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:49:36.925654 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.910743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.20.10.189:32775)
0807 22:49:36.911256 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.911260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.911276 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:49:36.911331 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.911335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.911408 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.911533 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.912476 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.913407 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.914687 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.915132 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.915290 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.919233 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.919242 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.919246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.919291 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.922580 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.923412 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.923418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.923421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.923569 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.923992 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.923998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.924001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.924384 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.924541 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.924939 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.925201 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":283,"threads_started":1}
I20260807 22:49:36.926956 13303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.188@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.10.189@KRBTEST.COM
I20260807 22:49:36.943887 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.927732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39516)
0807 22:49:36.927886 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.927905 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.928001 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.928563 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.928567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.929255 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.929462 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.929475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.930866 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.930871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.932107 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.932124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.933226 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.933232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.933453 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.934142 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.934162 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.936591 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.940582 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.940589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.940594 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.941268 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.941772 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.941779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.941783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.941972 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.942626 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.942631 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.942857 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.943052 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.943427 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:49:36.944073 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.927959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39516 (local address 127.20.10.188:40213)
0807 22:49:36.928194 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.928201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.928224 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:49:36.928283 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.928289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.928371 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.928493 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.929659 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.930716 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.932347 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.932941 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.933067 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.936830 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.936836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.936839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.936874 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.940269 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.941446 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.941452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.941454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.941621 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.942158 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.942162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.942164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.942510 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.942639 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.943132 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.943628 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":179,"server-negotiator.queue_time_us":75}
I20260807 22:49:36.945851 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.926509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34552 (local address 127.20.10.189:32775)
0807 22:49:36.926723 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.926729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.926747 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:49:36.926851 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.926855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.926931 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.927039 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.928879 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.930027 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.931759 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.932495 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.932689 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.938398 (+ 5709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.938405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.938408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.938450 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.942406 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.943477 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.943483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.943487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.943673 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.944230 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.944235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.944237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.944589 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.944888 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.945231 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.945493 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:49:36.950368 13303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: Bootstrap starting.
I20260807 22:49:36.951409 20522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:49:36.954098 13303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:36.955410 13303 log.cc:824] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:36.953915 13116 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54272
I20260807 22:49:36.958815 13303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: No bootstrap required, opened a new log
I20260807 22:49:36.966594 13303 raft_consensus.cc:348] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:36.967145 13303 raft_consensus.cc:729] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f947ea833b58493fb6febc56db0093e4, State: Initialized, Role: FOLLOWER
I20260807 22:49:36.967614 13223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 5
I20260807 22:49:36.967674 13303 consensus_queue.cc:260] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:36.968993 13312 sys_catalog.cc:455] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:36.969475 13312 sys_catalog.cc:458] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:36.970306 13303 sys_catalog.cc:564] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 22:49:36 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142976, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.189@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
I20260807 22:49:36.986264 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.968413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39530)
0807 22:49:36.968631 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:36.968652 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:36.968786 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:36.969315 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:36.969320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:36.969834 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:36.970004 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.970014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.971509 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.971514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.972808 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:36.972824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.973625 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.973632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:36.973916 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.974682 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:36.974713 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:36.977644 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:36.982509 (+ 4865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.982515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.982520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.983167 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.983943 (+ 776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:36.983951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:36.983956 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:36.984141 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:36.985138 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:36.985144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:36.985331 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 22:49:36.985508 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 22:49:36.985846 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:49:36.988530 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:36.968743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39530 (local address 127.20.10.188:40213)
0807 22:49:36.968960 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:36.968967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:36.968991 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:49:36.969051 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:36.969057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:36.969146 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:36.969248 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:36.970171 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.971358 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.972968 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:36.973454 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:36.973608 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:36.977887 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:36.977892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:36.977895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:36.977931 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:36.982320 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.983349 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.983358 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.983364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.983715 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:36.984333 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:36.984344 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:36.984350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:36.984861 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:36.985043 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:36.985745 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 22:49:36.986230 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 22:49:36.988507 13324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:49:36.988777 13324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:49:36.978896 13232 master_runner.cc:186] Error getting master registration for 127.20.10.190:39309: OK, instance_id { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" instance_seqno: 1786142975624540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:49:36.990060 13232 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:49:36.993942 13223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: Bootstrap starting.
I20260807 22:49:36.997344 13223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:36.998449 13223 log.cc:824] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:37.001430 13223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: No bootstrap required, opened a new log
I20260807 22:49:37.009116 13223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.009501 13223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c0590b98aa4ac79a56bd72f276bbc3, State: Initialized, Role: FOLLOWER
I20260807 22:49:37.010059 13223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.011150 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.011566 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.012396 13223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:49:37.024334 13340 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:49:37.024516 13340 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:49:37.020924 13152 master_runner.cc:186] Error getting master registration for 127.20.10.190:39309: OK, instance_id { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" instance_seqno: 1786142975624540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:49:37.031157 13152 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:49:37.065209 13144 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } attempt: 5
Aug 07 22:49:37 dist-test-slave-csg5 krb5kdc[13056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142975, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.10.190@KRBTEST.COM for kudu/127.20.10.188@KRBTEST.COM
I20260807 22:49:37.086047 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:37.066248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.20.10.188:40213)
0807 22:49:37.066488 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.066495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.066619 (+ 124us) server_negotiation.cc:407] Connection header received
0807 22:49:37.066890 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.066898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.066991 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.067104 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.068239 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.069468 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.071651 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.072501 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.072652 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:37.077504 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:37.077514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:37.077520 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:37.077578 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:37.082411 (+ 4833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.083542 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.083549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.083552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.083738 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.084237 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.084243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.084247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.084663 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:37.084822 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:37.085378 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 22:49:37.085654 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:49:37.086083 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:37.066083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.10.188:40213 (local address 127.0.0.1:39532)
0807 22:49:37.066507 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 22:49:37.066532 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:49:37.066740 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:49:37.067185 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:49:37.067192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:49:37.067803 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:49:37.068044 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:37.068059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.069673 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.069680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:37.071475 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:49:37.071495 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.072710 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.072719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:49:37.073001 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:37.073803 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:49:37.073831 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:49:37.077116 (+ 3285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:49:37.082681 (+ 5565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:37.082689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:37.082694 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:37.083355 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:37.083871 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:49:37.083877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:49:37.083881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:49:37.084093 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:49:37.084809 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:49:37.084815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:49:37.085018 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 22:49:37.085230 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 22:49:37.085617 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260807 22:49:37.093631 13144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4: Bootstrap starting.
W20260807 22:49:37.098995 13254 tablet.cc:2369] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:37.099977 13144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4: Neither blocks nor log segments found. Creating new log.
I20260807 22:49:37.101727 13144 log.cc:824] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:49:37.106488 13144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4: No bootstrap required, opened a new log
I20260807 22:49:37.116345 13144 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.116974 13144 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0effe12dfe94e69a6c40eddf1c22cd4, State: Initialized, Role: FOLLOWER
I20260807 22:49:37.117656 13144 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.119336 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.120298 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.120949 13144 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:49:37.121050 13116 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:49:37.121274 13116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:37.125205 13116 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.127197 13116 leader_election.cc:288] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [CANDIDATE]: Term 1 election: Requested vote from peers 20c0590b98aa4ac79a56bd72f276bbc3 (127.20.10.189:32775), f947ea833b58493fb6febc56db0093e4 (127.20.10.188:40213)
I20260807 22:49:37.128782 13196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "20c0590b98aa4ac79a56bd72f276bbc3"
I20260807 22:49:37.129401 13196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:37.133782 13196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0effe12dfe94e69a6c40eddf1c22cd4 in term 1.
I20260807 22:49:37.135035 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f947ea833b58493fb6febc56db0093e4"
I20260807 22:49:37.135637 13277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:49:37.139226 13089 leader_election.cc:302] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c0590b98aa4ac79a56bd72f276bbc3, a0effe12dfe94e69a6c40eddf1c22cd4; no voters:
I20260807 22:49:37.139771 13343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:49:37.140995 13277 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0effe12dfe94e69a6c40eddf1c22cd4 in term 1.
I20260807 22:49:37.152580 13343 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [term 1 LEADER]: Becoming Leader. State: Replica: a0effe12dfe94e69a6c40eddf1c22cd4, State: Running, Role: LEADER
I20260807 22:49:37.153481 13343 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } }
I20260807 22:49:37.163996 13347 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0effe12dfe94e69a6c40eddf1c22cd4. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.164639 13347 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:37.171888 13356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:49:37.185520 13356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 22:49:37.209769 13174 tablet.cc:2369] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:49:37.219630 13277 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [term 1 FOLLOWER]: Refusing update from remote peer a0effe12dfe94e69a6c40eddf1c22cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:37.221019 13196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [term 1 FOLLOWER]: Refusing update from remote peer a0effe12dfe94e69a6c40eddf1c22cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:49:37.221086 13347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:37.222519 13347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:49:37.228358 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:37.169086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.20.10.190:39309)
0807 22:49:37.169294 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.169302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.174407 (+ 5105us) server_negotiation.cc:407] Connection header received
0807 22:49:37.183351 (+ 8944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.183363 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.183504 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.183616 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.191030 (+ 7414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.192772 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.200461 (+ 7689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.201641 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.201808 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:37.215014 (+ 13206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:37.215023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:37.215030 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:37.215111 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:37.223589 (+ 8478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.224785 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.224794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.224802 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.225004 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.225618 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.225624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.225628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.226097 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:37.226436 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:37.227649 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0807 22:49:37.227948 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:49:37.242995 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:37.169298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.20.10.189:32775)
0807 22:49:37.169514 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.169521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.175258 (+ 5737us) server_negotiation.cc:407] Connection header received
0807 22:49:37.187192 (+ 11934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.187204 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.187350 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.187456 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.195840 (+ 8384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.197548 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.203579 (+ 6031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.204667 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.204835 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:37.218334 (+ 13499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:37.218344 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:37.218351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:37.218420 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:37.222607 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.239781 (+ 17174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.239790 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.239797 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.240026 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.240578 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.240583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.240585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.241060 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:37.241354 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:37.242010 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 22:49:37.242348 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:49:37.251060 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0effe12dfe94e69a6c40eddf1c22cd4. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.251430 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.265122 13369 mvcc.cc:205] Tried to move back new op lower bound from 7316041634647248896 to 7316041634427719680. Current Snapshot: MvccSnapshot[applied={T|T < 7316041634647248896}]
I20260807 22:49:37.267227 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.267583 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.269409 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:37.175643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39544 (local address 127.20.10.188:40213)
0807 22:49:37.175960 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.175969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.189428 (+ 13459us) server_negotiation.cc:407] Connection header received
0807 22:49:37.189942 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.189949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.190057 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.190132 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.199228 (+ 9096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.200365 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.202485 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:37.203410 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:37.203557 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:37.242949 (+ 39392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:49:37.242961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:49:37.242970 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:49:37.243041 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:49:37.251922 (+ 8881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.255024 (+ 3102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.255033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.255039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.255322 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:49:37.266861 (+ 11539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:49:37.266870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:49:37.266875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:49:37.267432 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:49:37.267778 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:37.268611 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 22:49:37.268966 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 22:49:37.270169 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.270592 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:37.283093 13312 sys_catalog.cc:455] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0effe12dfe94e69a6c40eddf1c22cd4. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.283460 13312 sys_catalog.cc:458] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.285862 13356 catalog_manager.cc:1347] Generated new cluster ID: c5affd217ab14300b5a8fa15bafced3b
I20260807 22:49:37.286010 13356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:49:37.291221 13370 mvcc.cc:205] Tried to move back new op lower bound from 7316041634647248896 to 7316041634427719680. Current Snapshot: MvccSnapshot[applied={T|T < 7316041634647248896}]
I20260807 22:49:37.294752 13347 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.295147 13347 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:49:37.305833 13312 sys_catalog.cc:455] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0effe12dfe94e69a6c40eddf1c22cd4" member_type: VOTER last_known_addr { host: "127.20.10.190" port: 39309 } } peers { permanent_uuid: "20c0590b98aa4ac79a56bd72f276bbc3" member_type: VOTER last_known_addr { host: "127.20.10.189" port: 32775 } } peers { permanent_uuid: "f947ea833b58493fb6febc56db0093e4" member_type: VOTER last_known_addr { host: "127.20.10.188" port: 40213 } } }
I20260807 22:49:37.306133 13312 sys_catalog.cc:458] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:49:37.323545 13356 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:49:37.325762 13356 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:49:37.351704 13356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0effe12dfe94e69a6c40eddf1c22cd4: Generated new TSK 0
I20260807 22:49:37.352947 13356 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:49:37.517810 13309 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.493654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.20.10.190:39309)
0807 22:49:37.493878 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.493885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.515380 (+ 21495us) server_negotiation.cc:407] Connection header received
0807 22:49:37.515973 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.515981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.516081 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.516160 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.517483 (+ 1323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38656: BlockingRecv error: recv got EOF from 127.0.0.1:38656 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 22:49:37.521045 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.500715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.20.10.189:32775)
0807 22:49:37.500951 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.500957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.518773 (+ 17816us) server_negotiation.cc:407] Connection header received
0807 22:49:37.519281 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.519287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.519392 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.519473 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.520787 (+ 1314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39656: BlockingRecv error: recv got EOF from 127.0.0.1:39656 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 22:49:37.524833 13306 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.522415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.20.10.188:40213)
0807 22:49:37.522626 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.522633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.523103 (+ 470us) server_negotiation.cc:407] Connection header received
0807 22:49:37.523375 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.523381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.523472 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.523549 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.524549 (+ 1000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35850: BlockingRecv error: recv got EOF from 127.0.0.1:35850 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 22:49:37.577651 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.564162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.20.10.189:32775)
0807 22:49:37.564380 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.564387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.575201 (+ 10814us) server_negotiation.cc:407] Connection header received
0807 22:49:37.575837 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.575845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.576023 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.576107 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.577336 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39664: BlockingRecv error: recv got EOF from 127.0.0.1:39664 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 22:49:37.628096 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.614437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.20.10.189:32775)
0807 22:49:37.614647 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.614684 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.625636 (+ 10952us) server_negotiation.cc:407] Connection header received
0807 22:49:37.626274 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.626282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.626404 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.626497 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.627808 (+ 1311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39670: BlockingRecv error: recv got EOF from 127.0.0.1:39670 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 22:49:37.685410 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.676877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.20.10.189:32775)
0807 22:49:37.677096 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.677103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.683411 (+ 6308us) server_negotiation.cc:407] Connection header received
0807 22:49:37.683635 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.683641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.683753 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.683891 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.685058 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39680: BlockingRecv error: recv got EOF from 127.0.0.1:39680 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 22:49:37.697417 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.694778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.20.10.189:32775)
0807 22:49:37.695060 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.695068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.695695 (+ 627us) server_negotiation.cc:407] Connection header received
0807 22:49:37.695933 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.695945 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.696052 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.696171 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.697168 (+ 997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39690: BlockingRecv error: recv got EOF from 127.0.0.1:39690 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 22:49:37.716337 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.713549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.20.10.189:32775)
0807 22:49:37.713796 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.713804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.714528 (+ 724us) server_negotiation.cc:407] Connection header received
0807 22:49:37.714832 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.714838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.714946 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.715053 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.716046 (+ 993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39694: BlockingRecv error: recv got EOF from 127.0.0.1:39694 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 22:49:37.739005 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.736620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.20.10.189:32775)
0807 22:49:37.736846 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.736853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.737267 (+ 414us) server_negotiation.cc:407] Connection header received
0807 22:49:37.737449 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.737457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.737564 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.737703 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.738723 (+ 1020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39704: BlockingRecv error: recv got EOF from 127.0.0.1:39704 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 22:49:37.779953 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.777013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.20.10.189:32775)
0807 22:49:37.777326 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.777334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.777758 (+ 424us) server_negotiation.cc:407] Connection header received
0807 22:49:37.778023 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.778027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.778221 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.778325 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.779639 (+ 1314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39716: BlockingRecv error: recv got EOF from 127.0.0.1:39716 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260807 22:49:37.851585 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.848682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39722 (local address 127.20.10.189:32775)
0807 22:49:37.848909 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.848916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.849594 (+ 678us) server_negotiation.cc:407] Connection header received
0807 22:49:37.849834 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.849839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.849928 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.850028 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.851318 (+ 1290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39722: BlockingRecv error: recv got EOF from 127.0.0.1:39722 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 22:49:37.985777 13310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:49:37.983582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.20.10.189:32775)
0807 22:49:37.983896 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:37.983902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:37.984188 (+ 286us) server_negotiation.cc:407] Connection header received
0807 22:49:37.984364 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:37.984370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:37.984476 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:37.984584 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:49:37.985496 (+ 912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39726: BlockingRecv error: recv got EOF from 127.0.0.1:39726 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 22:49:37.991047 13324 catalog_manager.cc:1259] Loaded cluster ID: c5affd217ab14300b5a8fa15bafced3b
I20260807 22:49:37.991155 13324 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: loading cluster ID for follower catalog manager: success
I20260807 22:49:37.993881 13324 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: acquiring CA information for follower catalog manager: success
I20260807 22:49:37.995839 13324 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f947ea833b58493fb6febc56db0093e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:49:38.026964 13340 catalog_manager.cc:1259] Loaded cluster ID: c5affd217ab14300b5a8fa15bafced3b
I20260807 22:49:38.027069 13340 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: loading cluster ID for follower catalog manager: success
I20260807 22:49:38.029691 13340 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: acquiring CA information for follower catalog manager: success
I20260807 22:49:38.031711 13340 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 20c0590b98aa4ac79a56bd72f276bbc3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:49:38.253206 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:38.246490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.20.10.189:32775)
0807 22:49:38.246766 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:38.246770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:38.246941 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:49:38.247070 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:38.247075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:38.247378 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:38.247491 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:38.247991 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.248835 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.250491 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.250996 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.251132 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:38.251922 (+ 790us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:38.252537 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:38.252660 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:38.252891 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 22:49:38.252980 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:49:38.267393 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:38.260482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.20.10.188:40213)
0807 22:49:38.261025 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:38.261030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:38.261051 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:49:38.261182 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:38.261186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:38.261386 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:38.261558 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:38.262039 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.262902 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.264422 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.264918 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.265036 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:38.265779 (+ 743us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:38.266492 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:38.266647 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:38.266928 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 22:49:38.267020 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":310,"threads_started":1}
I20260807 22:49:38.283977 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:38.274534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.20.10.190:39309)
0807 22:49:38.275188 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:38.275195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:38.275554 (+ 359us) server_negotiation.cc:407] Connection header received
0807 22:49:38.275715 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:38.275725 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:38.276013 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:38.276254 (+ 241us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:38.277236 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.278473 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.280480 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.281222 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.281360 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:38.282253 (+ 893us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:38.282845 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:38.282985 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:38.283487 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 22:49:38.283597 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":637,"threads_started":1}
I20260807 22:49:38.284015 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:38.275653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.20.10.188:40213)
0807 22:49:38.275902 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:38.275911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:38.276703 (+ 792us) server_negotiation.cc:407] Connection header received
0807 22:49:38.276992 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:38.277001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:38.277146 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:38.277261 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:38.278110 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.279381 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.281630 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.282380 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.282517 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:38.283133 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:38.283289 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:38.283411 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:38.283617 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 22:49:38.283719 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:49:38.284175 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:49:38.274918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39732 (local address 127.20.10.189:32775)
0807 22:49:38.275133 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 22:49:38.275140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:49:38.277075 (+ 1935us) server_negotiation.cc:407] Connection header received
0807 22:49:38.277267 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:49:38.277273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:49:38.277376 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:49:38.277482 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:49:38.278270 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.279529 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.281899 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:49:38.282545 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:49:38.282728 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:49:38.283309 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:49:38.283453 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:49:38.283554 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:49:38.283782 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 22:49:38.283870 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 22:49:38.290733 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 13070
I20260807 22:49:38.292088 13138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:38.413010 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 13070
W20260807 22:49:38.441603 13169 connection.cc:535] server connection from 127.0.0.1:34540 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:49:38.442097 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 13152
I20260807 22:49:38.443502 13218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:38.563392 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 13152
I20260807 22:49:38.590879 20522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu pid 13232
I20260807 22:49:38.592130 13298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 22:49:38.709167 20522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKr6fn6/build/asan/bin/kudu with pid 13232
2026-08-07T22:49:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5246 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10537 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (418415 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST